Please don't remove your labels. I know there seems to be this growing disdain for labels lately, to the point that the word "labels" is kind of becoming a buzzword of late, but labels aren't necessarily bad, or good. They're just descriptive terms. It's the stigma behind them which actually bothers people, for the most part. That and some people just happen to be so unique that no labels will actually stick to them very well.
I see a lot of people who gripe about how we don't need labels. I beg to differ. One of my labels is "gay." I find it significantly easier to state that I'm gay than to say that I am a man who is physically and emotionally attracted to other men. Why say, 10 or more words when I could say one? The label is far more efficient. And what reason would I have for getting rid of that label? I really can't think of any good that would come from that unless someone simply thought that there was some stigma attached to being gay. But really, that stigma is attached to men who are attracted to men, not to the word "gay."
Sometimes it's difficult to find a label that works for someone. I've noticed that in cases where you're dealing with a transgender person, labels can be confusing because each person will find a different method for applying a label in these instances, and that creates some confusion. So, many times, it would actually be more efficient to forgo the labels in favor of a more lengthy explanation of identities and orientations. That way there's no confusion.
I recently got into an argument about labels with someone whom I thought, at the time, was stating that we shouldn't need labels for most of the orientations and identities for which we have them. As the conversation progressed it became clearer what her opinion was and where the understanding of things was lacking. She was under the impression that "cisgender," a term for someone whose gender identity matches the sex to which they were assigned at birth (the opposite of transgender), was the same as saying that someone is "straight." Someone who is straight is simply someone who is attracted to the opposite sex. Their gender identity is a separate matter. This does make me wonder if some of the disdain, though, is due to people's lack of understanding as to the meanings of various labels, and they're afraid to ask.
The "bisexual" label is probably among the most controversial, with some people believing that bisexual people don't exist, and others believing they are just indecisive or greedy. Some people are quick to apply the "bi" label to someone who has otherwise been straight, but had one encounter with someone of the same sex. It might be premature to apply that label in that instance, as this could be a one-time thing, or it could be the discovery of a new aspect to this person's sexuality. No one knows but that individual. I don't personally think someone is bisexual if they've always been attracted to one sex and then had one moment where they were attracted to someone of another sex and it never happened again. That seems more like a fluke to me than an orientation. If they were attracted to both sexes on a regular basis throughout their life, then I would likely consider them bisexual.
We do also have to consider the fact that what labels we imagine should be applied to a person may not be the labels which they would apply to themselves. Case in point, a closeted gay man would appear to all outsiders as "straight," while he considers himself gay. Someone who sees a woman dating another woman might label her as a "lesbian," while she might consider herself bisexual.
I think labels are most accurate when they're based on the person's own feelings about themselves, which means that one's personal labels should be respected far more than what others feel are appropriate. Though sometimes people aren't honest with themselves, and might call themselves "straight" when they are actually gay or bi, and outsiders might see this and wish to use the "bi" or "gay" labels with them, and they'd be more accurate, but it's disrespecting that person's feelings. I'm a bit torn on this one. Is it better to let someone lie to themselves, or is it better to call a spade a spade? I don't know. If that person isn't ready to face the fact that they're bi or gay, trying to force it on them isn't going to help anyone.
Now, even though labels can be confusing at times, they're more useful than lengthy explanations are. It can be hard to know what labels to apply, though, especially when there are so many of them that some of them aren't even in common enough usage for them to be useful when applied. What use is the term "androphilic" if no one knows what it means? The simple labels, however, of "gay," "straight," "lesbian," "bi," or "trans" are all well understood, for the most part, and fairly easy to apply. If someone is uncomfortable with one of those labels, I doubt it's the word that is the problem, but the weight of it, what it signifies, and all stigma which is attached, which really is bothering them. We need to get to a point, though, where words like "gay" or "lesbian" are no more jarring than saying someone is a brunette or redhead. That would be ideal. They really are just descriptive terms, after all, which indicate where your preferences currently lie. Why that should be such a big deal, I don't know.
If you happen to be one of those people who has a tough time applying a label to yourself, please don't project those feelings onto others and disregard all labeling due to your frustrations with it. Find a way to describe yourself, while still respecting other people's right to label themselves as they choose. There's no obligation to have a label, but there's nothing wrong with having one, either. Also, beware that some labels can change over time, so try not to apply them too securely.