Discussion about this post

User's avatar
Balint's avatar

I'm definitely not an expert, but I'm not sure the conclusion is quite right here - my sense is that LLMs will generally give you the median position of news sources and educated elites (like you suggest) — UNLESS it thinks you might believe something else, in which case it tries to agree with you. This paper from Anthropic shows this kind of 'sycophancy' clearly (result of RLHF trying to maximise human preference scores: https://www-cdn.anthropic.com/e4f69aacd8c0905030172bc6eb480c252ea7d6ad/model-written-evals.pdf#page=28 (see section 4 espeically). Agree that from your prompts alone it is surprising how vehement its responses were though! perhaps the initial prompt/slightly leading follow-ups were enough, or perhaps it has remembered other info aobut you from other chats?

Expand full comment
Varut Subchareon's avatar

your first thought was also mine exactly. That ChatGPT is basically echoing what most of us think and feel. My immediate concern was also that they would somehow alter it. Very interesting experiment though. Really got my head spinning and really thinking about where were are in history. Sometimes I find it hard to really believe that we live in a time where we have technology like this.

Expand full comment
4 more comments...

No posts