If someone quotes Leviticus about sexual purity but ignores its commands about truth, wages, fair dealing, justice, strangers, slander, and honest measurements, they are not defending Leviticus. They are editing it.
People may say they do not judge, but they do. We all do. We notice clues, then we start building a story. That may sound harsh, but it is human. The question is not whether people should judge you. The question is what they are already seeing when they look in your direction.