You were wise to hold your tongue. He was trying to goad you into remarks he could hold you in contempt for...
When I lived in Dothan, I sat on a jury in a trial of a guy accused of molesting his 8 yr old step-daughter.
I was among three of the twelve jurors who held out for conviction. After 2-3 days of deliberation, we couldn't come up with a verdict.
On the morning of the 3rd day, we the jury sat in an empty courtroom. The judge came in and berated us for wasting the court's time and money, pointedly looking at those of us who thought the guy was guilty.
I risked contempt after that jury was dismissed. I went up to his office and sat down and told him that I and the other jury members were within our rights to vote to convict or acquit as our conscience sees fit. And he as the judge should only rule on thefairness of the case.
Luckily, I was polite enough and he had gotten over his anger. I was just ....ed off because I had tried to get out of jury duty (small children, and wife was out-of-town working) due to family obligations, and I had just spent 3 days in a jury box that I really didn't want to be in. And to have that guy talk to us like we were spoiled children. Just set me off.
I later found out that the accused was part of a good-old-boy network in Dothan. Everybody knew him, the D.A. and even the judge....who should have recused himself. They didn't want the case to go to trial in the 1st place. Small town justice.