However, let me point out from the side of someone who actually likes it.
I can make the distinction between a character that doesn't exist and a real child. Most everyone who draws/likes it can. I know someone who likes lolicon and teaches preschool children, and she hates them all with a passion. Because there's a distinction.
That's why it's a fetish. Most people can't help what they find arousing, be it furry, lolicon, or hell, even handcuffs and blindfolds. But if someone say, finds handcuffs and blindfolds arousing, that doesn't mean they're going to walk around in handcuffs or try to blindfold people.
What you're saying is that people who draw it have no self-control or distinction to real-world children. However, that's simply not true. Take Oneechan or Desuchan for example. Two sites that put lolicon up almost every day. I've been on both these sites for a long time, and only once was there a "true pedophile". He was immediately banned.