Optimal Model Activation Policies for Inference Networks of Large Language Models

Recent advances in large language models (LLMs) have rendered them necessary for Natural Language Processing (NLP) tasks, and their high inference cost…

Recent advances in large language models (LLMs) have rendered them necessary for Natural Language Processing (NLP) tasks, and their high inference cost motivates the study of cost-performance trade-offs. In practice, several expert LLMs are used in synergy for inference, either…

Read the original source — arxiv.org

paper · Shared by tscosj

0 comments

No comments yet.