# Content Aggregation Service API Documentation

This document provides information about the available API endpoints for the content aggregation service.

## Authentication

All API endpoints require authentication using a secret key. Include the secret key in the request headers:

```
X-Secret-Key: your_secret_key_here
```

## Endpoints

### 1. Fetch URL Content

Fetches and analyzes content from specified URLs.

- **URL:** `/fetchUrl`
- **Method:** `POST`
- **Request Body:**
  - `successful_insertions`: Array of URL objects to fetch
  - `feed_id`: ID of the feed these URLs belong to

### 2. Fetch RSS Feed Entries

Fetches and analyzes articles from a specified RSS feed URL.

- **URL:** `/fetchRSSEntries`
- **Method:** `POST`
- **Request Body:**
  - `feedUrl` (required): The URL of the RSS feed to fetch
  - `force_crawl` (optional): Boolean flag to force re-fetching of already processed entries
  - `skip_ai` (optional): Boolean flag to skip AI processing of content
  - `categories` (optional): Array of custom categories to use for article classification instead of the default categories

**Example Request Body:**
```json
{
  "feedUrl": "https://example.com/feed.xml",
  "force_crawl": false,
  "skip_ai": false,
  "categories": ["Technology", "Science", "Health", "Custom Category"]
}
```

The AI will classify each article into up to 3 of the provided categories. If no categories are provided, the system will use the default categories defined in the application.

### 3. Fetch Entry by URL

Retrieves an entry by its URL.

- **URL:** `/api/entry`
- **Method:** `GET`
- **Query Parameters:**
  - `url` (required): The URL of the entry to fetch

**Example Request:**
```
GET /api/entry?url=https://example.com/article
```

**Example Response:**
```json
{
  "id": "entry_123456789",
  "feed_id": "feed_123456789",
  "url": "https://example.com/article",
  "title": "Example Article",
  "description": "This is an example article",
  "smart_summary": "A concise summary of the article...",
  "smart_videos": [],
  "smart_images": ["https://example.com/image1.jpg"],
  "featured_image_url": "https://example.com/featured.jpg",
  "links": ["https://example.com/related"],
  "ttr": 120,
  "smart_quotes": ["Important quote from the article"],
  "last_crawl_success": "2023-06-15T10:30:00Z",
  "last_crawl_attempt": "2023-06-15T10:30:00Z",
  "favicon": "https://example.com/favicon.ico",
  "crawl_error": null,
  "rating_evergreen": 75,
  "rating_title_quality": 85,
  "rating_tone_balance": 80,
  "rating_research_depth": 70,
  "rating_grammar_quality": 90,
  "rating_thoughtfulness": 75,
  "rating_human_written": 85,
  "rating_humour": 60,
  "rating_originality": 70,
  "rating_fact_density": 80,
  "rating_sentiment": 75,
  "seo_buzzword_abuse_score": 15,
  "seo_trending_exploitation_score": 25,
  "seo_event_speculation_score": 10,
  "seo_clickbait_structure_score": 20,
  "seo_keyword_stuffing_score": 5,
  "seo_regurgitation_score": 30,
  "seo_timing_score": 15,
  "seo_content_manipulation_score": 10,
  "seo_spam_analysis": "Low spam likelihood. Content shows genuine insights with minimal clickbait structure.",
  "sentiment_label": "positive",
  "article_type": "analysis",
  "detected_language": "en",
  "has_paywall": false,
  "web_monetized": false,
  "smart_title": "Analysis of Current Technology Trends",
  "paywall_confidence": "low"
}
```

### 4. Search Entries by Keyword

Searches for entries containing a specific keyword in their title, description, or summary.

- **URL:** `/api/search`
- **Method:** `GET`
- **Query Parameters:**
  - `keyword` (required): The keyword to search for
  - `limit` (optional, default: 20): Number of results to return
  - `offset` (optional, default: 0): Number of results to skip

**Example Request:**
```
GET /api/search?keyword=technology&limit=10&offset=0
```

**Example Response:**
```json
{
  "entries": [
    {
      "id": "entry_123456789",
      "title": "Latest Technology Trends",
      "description": "An overview of the latest technology trends",
      "smart_summary": "This article discusses AI, blockchain, and quantum computing...",
      ...
    },
    ...
  ],
  "pagination": {
    "total": 45,
    "limit": 10,
    "offset": 0
  }
}
```

### 5. Fetch Entries by Feed URL

Retrieves all entries from a specific feed.

- **URL:** `/api/feed/entries`
- **Method:** `GET`
- **Query Parameters:**
  - `feedUrl` (required): The URL of the feed
  - `limit` (optional, default: 20): Number of results to return
  - `offset` (optional, default: 0): Number of results to skip

**Example Request:**
```
GET /api/feed/entries?feedUrl=https://example.com/feed.xml&limit=10&offset=0
```

**Example Response:**
```json
{
  "feed_id": "feed_123456789",
  "entries": [
    {
      "id": "entry_123456789",
      "title": "First Article",
      "description": "Description of the first article",
      ...
    },
    ...
  ],
  "pagination": {
    "total": 30,
    "limit": 10,
    "offset": 0
  }
}
```

### 6. Get All Feeds

Retrieves a list of all feeds.

- **URL:** `/api/feeds`
- **Method:** `GET`

**Example Response:**
```json
[
  {
    "id": "feed_123456789",
    "url": "https://example.com/feed.xml",
    "title": "Example Feed",
    "description": "A feed of example articles",
    "domain": "example.com",
    "favicon": "https://example.com/favicon.ico",
    "last_crawl_success": "2023-06-15T10:30:00Z",
    "last_crawl_attempt": "2023-06-15T10:30:00Z"
  },
  ...
]
```

### 7. Get Recent Entries

Retrieves the most recent entries across all feeds.

- **URL:** `/api/recent`
- **Method:** `GET`
- **Query Parameters:**
  - `limit` (optional, default: 20): Number of results to return
  - `offset` (optional, default: 0): Number of results to skip

**Example Request:**
```
GET /api/recent?limit=10&offset=0
```

**Example Response:**
```json
{
  "entries": [
    {
      "id": "entry_123456789",
      "title": "Recent Article",
      "description": "Description of a recent article",
      "feed_title": "Example Feed",
      "domain": "example.com",
      ...
    },
    ...
  ],
  "pagination": {
    "total": 100,
    "limit": 10,
    "offset": 0
  }
}
```

### 8. Get Entries by Domain

Retrieves entries from a specific domain.

- **URL:** `/api/domain/entries`
- **Method:** `GET`
- **Query Parameters:**
  - `domain` (required): The domain to fetch entries for
  - `limit` (optional, default: 20): Number of results to return
  - `offset` (optional, default: 0): Number of results to skip

**Example Request:**
```
GET /api/domain/entries?domain=example.com&limit=10&offset=0
```

**Example Response:**
```json
{
  "domain": "example.com",
  "entries": [
    {
      "id": "entry_123456789",
      "title": "Domain-specific Article",
      "description": "Description of a domain-specific article",
      "feed_title": "Example Feed",
      ...
    },
    ...
  ],
  "pagination": {
    "total": 25,
    "limit": 10,
    "offset": 0
  }
}
```

### 9. Get Statistics

Retrieves statistical information about the aggregated data.

- **URL:** `/api/stats`
- **Method:** `GET`

**Example Response:**
```json
{
  "feeds": 15,
  "entries": 450,
  "domains": 10,
  "recent_crawls": {
    "total": 50,
    "success": 48,
    "error": 2
  },
  "last_updated": "2023-06-15T12:00:00Z"
}
```

### 10. Get All Categories

Retrieves all available categories and their article counts.

- **URL:** `/api/categories`
- **Method:** `GET`
- **Query Parameters:**
  - `categories` (optional): JSON array of custom categories to use instead of defaults

**Example Request:**
```
GET /api/categories
```

**Example Response:**
```json
[
  {
    "name": "Technology",
    "count": 42
  },
  {
    "name": "Science",
    "count": 18
  },
  {
    "name": "Health",
    "count": 7
  }
]
```

### 11. Get Entries by Categories

Retrieves entries that match specific categories.

- **URL:** `/api/categories/entries`
- **Method:** `GET`
- **Query Parameters:**
  - `categories` (optional): JSON array of categories to filter by
  - `limit` (optional, default: 20): Number of results to return
  - `offset` (optional, default: 0): Number of results to skip

**Example Request:**
```
GET /api/categories/entries?categories=["Technology","Science"]&limit=10&offset=0
```

**Example Response:**
```json
{
  "categories": ["Technology", "Science"],
  "entries": [
    {
      "id": "entry_123456789",
      "title": "AI Breakthrough in Quantum Computing",
      "description": "A new AI algorithm improves quantum computing efficiency",
      "categories": ["Technology", "Science"],
      ...
    },
    ...
  ],
  "pagination": {
    "total": 15,
    "limit": 10,
    "offset": 0
  }
}
```

### 12. Health Check

Check the health status of the API services.

- **URL:** `/healthcheck`
- **Method:** `GET`

### 13. Database Health Check

Check the health status of the database connection.

- **URL:** `/db-health`
- **Method:** `GET`

## Error Handling

All endpoints return appropriate HTTP status codes:

- `200 OK`: Request succeeded
- `400 Bad Request`: Missing or invalid parameters
- `401 Unauthorized`: Invalid or missing secret key
- `404 Not Found`: Requested resource not found
- `500 Internal Server Error`: Server-side error

Error responses include a JSON object with an error message:

```json
{
  "error": "Error message",
  "details": "Detailed error information (if available)"
}
```