It's perfectly fine if you guys refuse to learn Japanese.
What really bothers me are foreigners who choose to live in Japan, and won't even bother learning the language at all.
I think those are the ones that really should learn the language, or just find a different country where you can use whatever language you already speak.