Model Endpoints and Servers

A Model Endpoint is the way to obtain model inference results in real-time, over the network.

There are 2 concepts that are closely related: Model Endpoint and Model Server.

Model Endpoint enforces certain network requirement on Model Server, while the Model Server exposes the underlying Model Version to network requests. The former defines the network interface, the latter controls the translation of network request to model input as well as model output to network response.