In the merged traces, the rows of question C05 carry two different prompt_sha256 values: one operator added a hint to the user template before running their model. What does the TD 1 protocol say?

Question

Answer

Delete the rows that carry the old fingerprint and keep only the latest attempt of each cell.

Answer

Keep all the rows in one experiment: the hint only makes the question clearer for that model.

Answer

Average the values obtained with the two prompts to smooth out the difference.

Answer

A changed prompt starts a separate experiment, with new `--state` and `--out` files and the old files kept: otherwise the gap is no longer attributable to the model.

Crowdly

In the merged traces, the rows of question C05 carry two different prompt_sha256...

Want instant access to all verified answers on learning.devinci.fr?