Language fashions can clarify neurons in language fashions

Though the overwhelming majority of our explanations rating poorly, we imagine we are able to now use ML strategies to additional enhance our capability to supply explanations. For instance, we discovered we have been capable of enhance scores by:

  • Iterating on explanations. We are able to enhance scores by asking GPT-4 to provide you with doable counterexamples, then revising explanations in gentle of their activations.
  • Utilizing bigger fashions to provide explanations. The common rating goes up because the explainer mannequin’s capabilities enhance. Nonetheless, even GPT-4 offers worse explanations than people, suggesting room for enchancment.
  • Altering the structure of the defined mannequin. Coaching fashions with totally different activation capabilities improved rationalization scores.

We’re open-sourcing our datasets and visualization instruments for GPT-4-written explanations of all 307,200 neurons in GPT-2, in addition to code for rationalization and scoring using publicly available models on the OpenAI API. We hope the analysis group will develop new strategies for producing higher-scoring explanations and higher instruments for exploring GPT-2 utilizing explanations.

We discovered over 1,000 neurons with explanations that scored a minimum of 0.8, which means that in keeping with GPT-4 they account for a lot of the neuron’s top-activating conduct. Most of those well-explained neurons usually are not very fascinating. Nonetheless, we additionally discovered many fascinating neurons that GPT-4 did not perceive. We hope as explanations enhance we could possibly quickly uncover fascinating qualitative understanding of mannequin computations.

Date: 2023-05-09 03:00:00

Source link



Related articles

Alina A, Toronto
Alina A, Toronto
Alina A, an UofT graduate & Google Certified Cyber Security analyst, currently based in Toronto, Canada. She is passionate for Research and to write about Cyber-security related issues, trends and concerns in an emerging digital world.


Please enter your comment!
Please enter your name here