Well it's happened twice so far. Granted, the carrier I use is a chocolate brown, and the stroller and car seat are a neutral tan color, but when the child is draped in a bright pink blanket and wearing a headband, do you really have to ask?
The first time it happened was at a social gathering. She was wearing a brown onesie with a cute kitten and pink ball of yarn on the front. Her pants were pink with brown polka dots, and Husband was holding her pink blanket under her. The elderly gentleman commented on what a cute baby it was, and then asked if it was a boy or girl. Husband and I immediately said girl, and the gentleman looked a little offended. Then he took a closer look at her and corrected himself; "Oh, I guess she is wearing pink."
The second time it happened I was at the mall sitting on a bench with her. She was wearing a turquoise outfit; a turquoise and purple onesie and matching pants. I had even found a perfectly matching turquoise headband with a little bow to go with it. An older lady came up to me and asked me how old my baby was. Then she told me her daughter was expecting a baby boy in a few weeks. Then she asked if my baby was a boy. Not boy or girl. Just boy. After I told her my baby was a girl, she noticed the headband. I told her that's why I put the headband on her, so it's easier to tell.
Is it really that hard to tell the difference between a boy baby and girl baby? I can understand if the baby is wearing completely neutral colors, with all neutral items nearby, but if there are specific colors involved, just take an educated guess. I can see why we have stereotypes and make colors specific to one gender or another. However, there are still some people who just don't get it. Plus, who would put a headband with a bow on a baby boy?