An LLM deployed in a real-time recommendation system is having noticeable slowdowns during peak usage times. What would be the most effective approach to solve this performance issue?

Question

Answer

Enable batch processing for inference to handle multiple requests simultaneously

Answer

Migrate the application to a server with higher CPU capacity

Answer

Implement model quantization of the LLM

Answer

Increase the number of layers in the model to enhance its decision-making capabilities

Crowdly

An LLM deployed in a real-time recommendation system is having noticeable slowdo...

Want instant access to all verified answers on softserve.academy?