llama.cpp b11000

rpc : invalidate cached compute graph when a referenced buffer is freed ( #24292 ) The server caches the most recent compute graph per device so that…

rpc : invalidate cached compute graph when a referenced buffer is freed ( #24292 ) The server caches the most recent compute graph per device so that GRAPH_RECOMPUTE can re-execute it without resending tensor data. The cached graph nodes hold direct pointers to backend buffers…

Read the original source — github.com

release · Shared by tscosj

0 comments

No comments yet.