Would you be annoyed if your brother-in-law started yelling at you?

My brother in law started yelling at my wife and I because our 4 year old took a quarter from his 3 years old and ran with it. He said we didn't do anything. The issue with this is that our son ran and we had no idea what he had in his hand. Apparently my bil feels that we don't intervene and that our son picks on his daughter. They pick on each other and when he does we say something. We don't ever pretend to act like our son is perfect. He is 4! His daughter always says no to us when we tell her to get off of our tables, sofas, counters, ect. my bil does nothing. When she takes toys and things from our kids and we say something, he says nothing. In fact a few months ago his daughter threw dirt in our sons eyes and he asked her if she got him good! THis really pisses us off that he thinks we don't do anything when he sits on his *** and watches his daughter destroy things, hit kids, and take things from other kids, then acts like because she is the youngest everyone is suppose to baby her! What is your opinion of this?