I would say yes, wholeheartedly! I feel like I notice a lot about my restaurant surroundings, not just the food. One big thing that can make me yea or nay a restaurant is acoustics. It doesn't seem to bother some people at all, but I do not enjoy a meal if I have to really concentrate and strain to hear my companions, and feel like I'm yelling in order to be heard myself. The problems I find are loud music, poorly placed speakers, and no sound buffering in small spaces. I wonder why it bothers some people but not others? My dad has the same complaint as I do, but he's hard of hearing and my hearing is actually a little better than average, so it covers a broad spectrum.
All the more subtle details contribute as well, of course. Like,
Good: ambient, warm, diffuse, medium/low brightness
Bad: harsh, cool, bright, overhead
Good: simple, uncluttered, harmonious but not totally matchy-matchy, exhibits unique character, no stark white anywhere unless it's the linens
Bad: fake plants, that dentist's office waiting room look with medium/pale blues and mauves everywhere and boring "art" posters in shiny faux gold frames
Good: clean clothing with everyone in a similar color scheme (black top and pants look nice) and clean waist aprons.
Bad: overtly slutty outfits on women, head to toe identical uniforms (maybe it's just me but I think uniforms look absurd and degrading, like management is trying to partially remove the servers' identities), visibly dirty/stained clothing or apron
Cutlery and dinnerware
Good: 4 tined salad and dinner forks, well balanced cutlery, white or black dishes with little or no embellishment, clear drinking glasses so I can see if they're clean and see what I'm drinking
Bad: 3 tined salad/dinner forks, cutlery with excessively heavy handles, colored/decorative plates that distract from the food
Blah, blah, blah. But it also depends on my expectations. I love diners and inexpensive little Vietnamese places and I don't expect them to adhere to these standards, only finer dining venues.