PEGASUS uses Gap-Sentence Generation (GSG) as its pre-training task. Which statements are correct?

Question

Accepted Answer

Entire sentences are removed from the document and the model must generate them

Accepted Answer

The sentences chosen for removal are the most important ones, measured by overlap with the rest of the document

Answer

GSG masks individual words randomly, just like  BERT

Answer

This pre-training task closely resembles summarization, so the model needs less fine-tuning data

Crowdly