@tom_doerr - When I use an LLM as a judge, I ask a True/False question
Tom Dörr✓@tom_doerr
2025-01-15When I use an LLM as a judge, I ask a True/False question and do (p(True)+p(true)) - (p(False)+p(false)). This works very well for me, but I don't think I've seen anyone else scoring answers this way