Horror is probably one of the hardest genres of anything to write, and there are a lot of misconceptions regarding the actual process of it. The easiest way to explain it is to say that it's most basic objective is to both terrify and interest the reader. Manipulating fear is a skill, whether you can do it right off the bat, or require practice. Knowing how to scare people is the most basic element. But that is only half of the story. Horror stories still need a plot, because they will come off as bland, and quite utterly dumb if all you're trying to do is use imagery to scare people, without any particular direction. Sure, the images will be conjured in the reader's mind as scary, and probably are. To the writer. But they'll be incoherent, more disturbing and thought-provoking than actually scary to the viewer. But here's the basics:
Number one: develop a plot. Yes, it's that simple. Horror is like any other genre; it needs a story. In most cases, it's quite mandatory. Even if you intend not to explain what is going on, there needs to be a flow. On this topic, there has to be a careful balance between the plot and the "horror". You can't have more horror than plot because that just turns it into the action/adventure genre, with a gorey or interesting twist. You can't have too much plot, because then it will turn into a mystery, especially if it's a serial killer involved. Getting the right balance depends on a few things. First, your experience. The inexperienced writer often goes for the "shock and awe" method. This is not what you want to do. It will come off as "badass" rather than scary. Most successful writers build up action and suspense before having their scene unfold.
Number Two: Study. Even if you don't need to, there is a certain way to scare people. And when you do scare people it also shows that you know how people work. You can play to their instincts, fears, and dark thoughts. This is the essence of horror. You use the inherent fear that lies within each person and pander to it. Regardless of what you choose to scare with, you must know how to use that. Know how people work. Pay attention to what scares your friends. Analyze why it scared them. Or even better, what scares you? Why does it scare you? This is what makes this type of writing successful. Once you know why something scares someone, you can effectively build up the suspense to it and scare them. The best and most reliable way to do this is study phobias, or irrational fears. Everyone has at least one, and chances are if you appeal to a phobia, you'll achieve quite an effect. But again, this is quite useless without knowing what it is about the phobia that scares people. Fear is often like a dream; full of symbolism. If someone is afraid of the dark, that can mean a few things, depending on what aspect of the dark scares them. They might actually be scared of being blind ("there are things in the dark trying to get me, and I can't see in the dark") or alone ("Night time makes the world seem so big and scary. The quiet makes it feel so empty"). Or maybe they are afraid of what they cannot see (A fear of the unknown, like perceiving a monster in the closet or under the bed). A fear of drowning can easily expand into the universal concept of death by suffocation or asphyxiation. Or, if it is a monster you're using, you might want to draw upon an animal phobia. Arachnophobia (the fear of spiders) is an all too common phobia that, while overplayed, can be used successfully. It doesn't have to be a gigantic tarantula that terrorizes people in their beds. It can be a bunch of tiny spiders. A common nightmare of mine is a writhing mass of gray spiders (think along the lines of a moving mass of shredded egg-carton). It's not too big and not too small. Most insect fears come from the fear of being bitten, their grotesque appearance, or just the fact that they have so many cold feet with which they tickle you in the most macabre way. Or, overplay the fear of spiders by using a few tarantulas. But this also gets us started on the next section.
Number three: Moderation. This is key. This is the balance I mentioned earlier. Any good or bad horror novel has suspense. Usually starting off with a macabre perspective of every day life, or some loser in school. But in any case, the suspense half of horror is also the plot. The plot forces the horror behind a tissue-paper curtain for a while. It's poorly disguised, still there, but not prevalent to the reader. Instead, while the "normal" stuff is going on, that thing behind the curtain continues to writhe and move, or become angry. You know something will happen, you just don't know what or when it will happen. Again, don't allow it to sit for too long, as this monster will become bored and eventually fall asleep behind the tissue paper curtain. When you reopen this curtain, you don't want your reader to find a peacefully sleeping animal. You want them to find a rabid werewolf with bad breath and a lust for blood tearing into the guts of some poor soul that's screaming his head off. Not some cute anthropomorphic creature feeding simply because you deprived him of his dog food for too long while you went into depth about how it works and what it's biology is and what it does during the day and night. By contrast, you don't want to let this monster out of the bag too often. This will probably result in your main character performing impressive feats such as running marathons, triathlons, and then getting themselves lost while an unfortunate trail of blood follows them. This will actually become quite annoying because the reader may start to wonder why this person is running if they are so athletic. They'll be perceived as a coward.
Number Four: Isolation. There are a few things that we could learn from Stephen King...one of his most prominent cliches is his use of isolation. Whether the main character really is alone or there is a group of people, the reader must feel that the characters are alone in their battle. Even if the world is not like the Stephen King universes where the main characters are the only normal people and everyone else is monumental, legendary jerk, there should be a sense of isolation. This can be as simple as ignoring the crowds around your characters, or making their story to others so unbelievable by rational standards that they have to bottle it up and handle the matter themselves. This also serves a higher purpose. As humans we are social creatures. Even those so-called "antisocial" types have to have regular contact. The only people that don't are either already terminally crazy or sociopaths. Nearly everyone needs some kind of contact with another human being. Taking that away or ignoring it creates the effect of loneliness, which when coupled with a fear stimulus, will create an amplified effect to the reader, turning a mildly creepy situation into a fight for their life. Think about it. If you have an angel that kills people, that picks one person out of the millions in New York, that's kinda creepy. But if it doesn't discriminate like it does, then it kills everyone. Being an angel, it is quite invincible and it ceases to be a story focused on one or a few characters, and becomes a story about the death of the human race. Instead of being scary, it's depressing. The cliché might be overplayed in many books, but it is quite necessary. Due to being overplayed, though, it's best if this aspect is not in the foreground of perspective. It should be subtle.
Number five: subtlety. Apart from alternate reality type stories, all of the above should be subtle, unless you're intending to write a creepypasta or a short story where everything has to be condensed down. This is different from moderation. Moderation applies to the frequency of events. Subtlety applies to the amplitude. This varies from writer to writer, as each person is capable of different effects. But keep in mind that a book is not like a movie. Having a scene where the villain or object of terror suddenly shows up in someone's room, dramatically grabbing their ankle and slicing off their foot or something like that will not have the same effect as it does on a movie screen. Instead of using pictures and audio to paint your story, you'll have to settle with words. Suspense and slowness is a much better tool. Having something with creepy eyes staring at you over the foot of your bed is much creepier than a dramatic scene where it just tears you apart Mad Butcher style. There's a time and a place in writing for those scenes; the climax. For the most part, the only time you should hear about such gruesome drama is in the background, such as the characters overhearing various news reports drifting in from the television or radio.
Number six: The right characters. Horror is not like an action story where any character can fit into almost any role. Depending on how well you write, you're going to have to choose characters you don't like. Writing a horror story for, say, a private investigator is extremely hard. Why? A PI's job is to know each and every detail. The same goes for cops and Special Agents. A better approach with such characters is to barely even mention the horror element and let it play itself out as the investigation continues all too slowly. By contrast, a night stalker visiting a twelve year old (imagine if Pedobear had come from the movie Poltergeist for this example) would happen nearly every night. In this case, we know what the element of horror is. This kind of thing should have an escalating effect where it starts with creepy stares, possibly glowing eyes and quiet laughter to a horrifying climax where you choose what happens. This twelve-year old is quite incapable of defending himself as this mad creation torments him. By contrast, some people prefer strong characters such as the aforementioned investigators who have training and are at least somewhat prepared to deal with the hardships ahead. In using this type of character, you must be very careful about what goes on. Since the character is strong enough to handle whatever is happening, the object of horror might then start tormenting his loved ones instead of the investigator himself. Although it is quite probable that the character being strong is an illusion. While he can protect himself, he cannot protect what is most precious to him.
Number Seven: Obscenity. I normally inhibit myself when writing, but in many cultures, there are certain taboos which are never mentioned for their cultural disgust. For many right now, it is pedophelia. Now, before you go off on your rant about how offensive that material is, let me tell you about the wonderful thing that is imagination. At this point, the scene is already set up. You can pan perspective off of the scene as the screams emanate from this poor soul (The pedobear scene mentioned not too long ago). The reader (right now being you) is most likely to fill in the void with their own assumption, which more likely, pertains to their own fears and will gain a more memorable impression upon the reader. This also leads us to the next section.
Number Eight: Anonymity. One of the mistakes people often make is adding too much description. This works for real world incidents, such as the horrific incidents in the Rwanda genocide because that was a truly horrific occurrence that is history making, and sometimes the best way to learn from history is to be shocked and disgusted by it. More often then not, adding that kind of obscene description tends to just make the author look insensitive and psychotic. No one in their right mind would write in such detail about such things as that. In fact, it may sound like the book is your own private fantasy about torture. There is a place for such description. For example, if your main character is actually the object of horror, and they are going quite insane, then some amount of description is necessary, but more about how they are enjoying themselves than what they are doing. This is what makes a lot of serial killers (who, in reality, should be called "spree killers", but that's another issue for another day...) scary. They feel no guilt. They flat out enjoy killing and torturing. You might even go so far as to make it sexual. Describing the things around the incident, or "panning off" of them can actually be more effective and ironically more appropriate because you're allowing your reader to fill in the blanks with their own dirty mind, instead of you filling theirs in for them. Giving a paragraph of description to something doesn't make it scary. It just makes it obscene, and can get you in trouble, especially in the areas that involve sex. There are some things that do not need a lot of description. By contrast, though, there are rare exceptions. Torture scenes naturally require some attention to detail. Some sadistic maniacs act like doctors and explain in gruesome detail what they are about to do to their victim...to their victim.
But most of the time, describe the sensory perceptions experienced, not the scene. On the note of description, however, most people have not smelled dead bodies or the like. You might want to read eyewitness accounts or other horror stories to find the right words to describe such things, even if minimally. Saying that something "smells like bile" may even confuse your reader. Bile in itself requires some research to know how it smells. Sure, we know it's disgusting, but do we really know what it smells like? Most of us don't even want to think about such disgusting things...and forcing your reader to think about it is some sadistic fun. Anonymity isn't really that difficult. It's kind of like writing in a way that reverses the role of reader and writer. You know what's going on...but the reader, being the screwed up person they are, will assume the worst. In some areas, a horror writer is only as screwed up as his audience.
Number eight: It's not scary. You say your audience isn't scared?! Something that scares some people will NOT scare someone else. Fears and phobias are not universal. I must mention that there is a relatively new phenomenon in comics and cinema: shock horror (also known as the trope "darker and edgier"). This shows images that border on being pornographic (Also known to some few as "gorn") to shock the viewer into being scared. I do not recommend this. Not only does it desensitize your audience, but it's really not scary. It's gross. That's my biggest gripe with slasher films. Don't confuse shock value with true fear. Now...horror, like everything, takes practice. Ideally, it should have some research backing it. If you choose something scary, understand why its scary, and then exploit that answer to the fullest. But if you're absolutely incapable of scaring someone (for example...you're that one innocent kid that gets laughed at when they get mad), don't be discouraged. Brush up your vocabulary for some more evocative words, and maybe even observe people objectively. Objective observation is what psychologists do on a daily basis, where they observe the stimulus and response of people's behavior. Heck, ask your audience what scares them, and then study the phobias applying to that fear. If you use these methods, you should be getting better. This is basically a list of what I've observed from reading horror. If I missed something, or you think you could do something better that's completely different, feel free to tell me...I'm no expert. I just observed all this and wrote about it.
And just one more final tip: Vocabulary should be used properly. Not too sophisticated, but not too dim, either. This is crucial to the area of description.