The colors really aren't bad at all - they're probably the strongest element of this piece. But I would ditch the linework ASAP. It's fine to start by sketching in the basic composition like that on a layer...I do that all the time. But unless you're going to go in and really meticulously perfect the line quality, hide those lines for the final painting. Define the edges with contrast between colors and textures, not lines.
Also, compositionally, I think this piece would be a lot better off if you cropped out the left side and added more to the right. Your house is "looking" off to our right, out of the frame. If a major object or person is going to be turned noticeably toward one side, I like to have it turned toward the center of the frame, not toward the outside. Helps keep the viewer's eye inside the painting.