Begin by imagining a hypothetical universe. In this hypothetical universe every fact about a person is recorded in some database somewhere everyone has access to. Actions people have done, character traits they have, everything. Pretend everyone has the ability to ask some really cool AI thinger any question about a person and receive statistically based answers. "What is the likelihood of Casey getting cancer?" "Casey has about a 13% chance of getting cancer in the next 8 years."
Now pretend I saw a listing on Classifieds for something. I find out who the person is, Sasha, and ask the AI, "What is the probability Sasha is scamming me?" "There is about a 76% chance Sasha is scamming you." Should I continue the interaction? Should I break it off and move on? If I get scammed was I acting with naive hope? If I break it off and Sasha isn't actually scamming me am I making a hurtful decision? What if the number was 2%, 90%, or 50%?
My opinion is it would be perfectly sensible to make statistically based initial assessments in this hypothetical universe. Depending on the likelihood in the scenario above I would break off the interaction. My percentage where I would start flipping is probably around 30-50%. It depends on how badly I wanted the item, the cost of being scammed, and how much I need the money I could potentially lose.
Imagine a different scenario in this hypothetical universe. Pretend I'm a security guard for a store of some kind. I have a digital display constantly asking the AI, "What is the probability that specific person walking in will steal from me?" At what percentage of likelihood would it be okay to take preemptive action and kick the person out? Or is it never okay to take preemptive action?
The reasons I'm good with making initial assessments, in this hypothetical universe.
1. This magical database knows every fact about every person. I can know, with certainty, the answers I'm getting are probabilities based on facts.
2. I am prepared to have my initial assessment be wrong, since we're talking about varying degrees of probability.
3. Everyone knows the odds, so in the scenario above Sasha could say, "What is the probability I'm scamming David?" "There is a 76% chance you are scamming David," the same answer back. So if I choose to break off the interaction Sasha would understand.
4. The AI thinger isn't making decisions for me. It's informing me of the odds and I get to make my own decisions.
Step back into our universe. We are missing many things from the hypothetical universe. No magical database of facts, no magical AI, and no equal knowledge of the statistical odds. Therefore, making initial assessments based on statistical probabilities is generally fraught with peril. There's a good chance the "facts" I'm basing my assessment on are others' opinions. My perspective could be not representative of the world at large. I probably don't really know the person I'm interacting with. The odds are probably not as high or low as I think they are.
There are cases in our universe where making an initial assessment based on statistics is logical. If I mention a professional League of Legends player you can assume, with 99% certainty, I'm talking about a male. If you mention a nurse friend who works in the United States I can assume, with 90.5% confidence, you're talking about a female. In these cases I'm fine having my initial assessment be wrong, because these are probabilities. Additionally, I believe examples like these are few and far between, for the reasons I just mentioned.
Another place where we get into trouble with statistically based initial assessments is when we apply valid, accurate statistics improperly. For example, in cases of shootings in the United States involving more than four victims the perpetrator has a 97% chance to be male. That statement by itself is accurate and based on historical evidence. Additionally, if I hear of a new shooting with four or more victims I can assume, with 97% odds, the shooter is male. However, a statement like, "because you're a male I should be worried about you shooting four or more people" is an improper application of the factual statistic cited above.
I have a person I work with I greatly respect. He is fond of saying, "data is not offensive." I wholeheartedly agree with that statement. Data given without proper context has the potential to lead folks down an illogical path. Also I've learned to be cautious about where I accept statistics from, and what I apply those statistics to. Having frank conversations about accurate data is powerful, and can lead to real positive change.
My conclusion is that making statistically based initial assessments in the hypothetical universe I constructed is perfectly logical. In our universe making statistically based initial assessments is rarely logical. Data is not offensive and discussions about it can lead to positive societal change.