First, you need to realize there are two different types of editing. There is line editing and plot editing (also called development editing).
Plot editing is the guts of your novel. It’s not only the plot itself, but your characterizations and dialogue, structure and narrative. A lot of it is your style of telling a story. This stuff is important because you want your plot to make sense logically to the reader, your characters need to stand out from one another and the characters’ dialogue needs to be appropriate and distinct; you don’t want all your characters to talk exactly the same because it’s boring to the reader. To add, your story structure needs to flow well, to keep your plot moving. Narrative needs to remain consistent. All of this will help the reader to enjoy their experience with your book all the more, and could have them wanting to see more work from you. Also, following these tips will make your writing appear strong to editors and publishers, and you want to look good to those people if you want to be a published novelist.
Line editing is the nuts and bolts work. Things like checking for grammar, spelling, punctuation, etc. A lot of this might seem tedious work, but it is important. A clean manuscript says a lot to editors and publishers to whom you are submitting your work. It tells them you are serious. It tells them you are professional. And it tells them you are someone they can probably work with. Before you reach this stage in your editing, you have possibly had to do a fair amount of rewriting, so line editing really should be the last type of editing you to do your manuscript.
For editing, especially line editing, I’ll suggest two books that can help:
The Elements of Style, by William Strunk Jr. and E. B. White
The Pocket Idiot’s Guide to Grammar and Punctuation, by Jay Stevenson, PhD.
These two books can be indispensable, and I suggest keeping them right next to your favorite writing spot along with a dictionary and thesaurus.
Breaking the Rules
Keep in mind there are times for writers to break the rules. For example, sometimes you might want to spell a word incorrectly for one reason or another, maybe because the story is told in first person and that’s how the character would spell it. Just remember that breaking the rules should only be done for a good reason, and usually is only done well by an experienced writer. Also, don’t forget that you might have to explain to an editor or publisher why you are breaking a certain rule or rules; and also remember that if you have to explain why you’re breaking a rule, then you probably shouldn’t be breaking the rule in the first case.