Model Training Comparison

{
  "type": "model-training-comparison",
  "name": "LLM Output Comparison",
  "description": "Review and compare the outputs from two different LLMs for the same prompt"
}

{
  "data": {
    "userPrompt": "What are the key differences between REST and GraphQL APIs, and when should you use each one?",
    "llmResponses": [
      {
        "name": "LLM A Response",
        "response": "## REST vs GraphQL\n\nREST (Representational State Transfer) and GraphQL are both API design approaches, but they differ in several key ways:\n\n**REST:**\n- Uses multiple endpoints (one per resource)\n- Fixed data structure - you get all fields from an endpoint\n- HTTP methods (GET, POST, PUT, DELETE) for operations\n- Stateless communication\n- Caching is easier due to HTTP standards\n\n**GraphQL:**\n- Single endpoint for all queries\n- Flexible - clients request only the data they need\n- Query language for fetching data\n- Strongly typed schema\n- Can reduce over-fetching and under-fetching\n\n**When to use REST:**\n- Simple CRUD operations\n- When you need HTTP caching\n- When working with file uploads/downloads\n- When your data model is straightforward\n\n**When to use GraphQL:**\n- Complex data relationships\n- Mobile apps needing to minimize data transfer\n- When clients have varying data requirements\n- Real-time updates with subscriptions"
      },
      {
        "name": "LLM B Response",
        "response": "## Understanding REST and GraphQL\n\nBoth REST and GraphQL solve the problem of API design, but take different philosophical approaches.\n\n**REST APIs:**\nREST follows resource-based architecture. Each URL represents a resource, and HTTP verbs define actions. For example:\n- GET /users - fetch all users\n- GET /users/123 - fetch user 123\n- POST /users - create a user\n\nThe server determines what data to return, which can lead to over-fetching (getting more data than needed) or under-fetching (requiring multiple requests).\n\n**GraphQL APIs:**\nGraphQL uses a single endpoint and a query language. Clients specify exactly what fields they want:\n\n```graphql\nquery {\n  user(id: 123) {\n    name\n    email\n  }\n}\n```\n\n**Key Trade-offs:**\n\nREST advantages:\n- Simpler mental model\n- Better browser caching\n- Easier to debug (standard HTTP)\n- More mature tooling\n\nGraphQL advantages:\n- Precise data fetching\n- Single request for related data\n- Self-documenting schema\n- Better for complex apps\n\n**Recommendation:**\nUse REST for simple, well-defined APIs. Use GraphQL when you have complex data relationships or need to optimize for mobile/bandwidth-constrained clients."
      }
    ]
  },
  "ui": {
    "userPrompt": {
      "ui:title": "User Prompt",
      "ui:description": "The original prompt/question submitted to both LLMs",
      "ui:widget": "markdown"
    },
    "llmResponses": {
      "ui:title": "LLM Responses",
      "ui:description": "Compare the following responses from two different LLMs",
      "ui:widget": "compare",
      "items": {
        "response": {
          "ui:widget": "markdown"
        }
      }
    }
  }
}

[
  {
    "title": "Submit Review",
    "description": "Select your preferred response and provide feedback",
    "id": "submit-review",
    "schema": {
      "type": "object",
      "required": [
        "preferred-response",
        "information-accuracy"
      ],
      "properties": {
        "preferred-response": {
          "type": "string",
          "title": "Select the response you liked best",
          "description": "Choose which LLM response you prefer",
          "enum": [
            "llm-a",
            "llm-b"
          ],
          "enumNames": [
            "LLM A Response",
            "LLM B Response"
          ]
        },
        "preference-reason": {
          "type": "string",
          "title": "Reason why you liked it best",
          "description": "Explain what made this response stand out to you"
        },
        "information-accuracy": {
          "type": "string",
          "title": "Is the information accurate",
          "description": "Does the selected response contain accurate information?",
          "enum": [
            "yes",
            "no"
          ],
          "enumNames": [
            "Yes",
            "No"
          ]
        }
      }
    },
    "ui": {
      "preferred-response": {
        "ui:widget": "radio"
      },
      "preference-reason": {
        "ui:widget": "textarea",
        "ui:placeholder": "Share your thoughts on why you preferred this response..."
      },
      "information-accuracy": {
        "ui:widget": "radio"
      }
    }
  }
]