Comet - Custom Niche

The Comet endpoint suite enables you to create custom niche configurations that automatically discover and organize videos based on your specific keywords and criteria. Set up recurring scraping jobs with flexible scheduling, platform targeting, and filtering options. Perfect for monitoring specific topics, competitors, or content themes over time.

POST/comet

Create comet

Creates a new custom niche configuration that will automatically scrape videos based on your provided keywords, platforms, and scheduling settings. The configuration runs on the specified cadence and collects videos matching your criteria.

Cost per request:

Request body

Name
name
Type
string
Description
A descriptive name for this configuration. Example: "Tech Reviews" or "AI Content Tracker"
Name
keywords
Type
array
Description
Array of keywords to search for (1-20 keywords). Example: ["tech review", "gadget review", "product review"]
Pro tip: Use specific multi-word phrases for better results. Avoid generic single words.
Name
platforms
Type
array
Description
Platforms to scrape from. Options: youtube, tiktok, instagram. Must include at least one platform.
Name
cadence
Type
string
Description
Schedule cadence. Options:
- Simple: "daily", "weekly", "monthly"
- Cron expression: Any valid cron (minimum daily frequency)
Examples: "daily" or "0 0 * * *" (midnight daily)
Name
min_views
Type
number
Description
Minimum view count threshold for videos. Videos below this threshold will be filtered out. Must be >= 0.
Name
time_range
Type
string
Description
Time period filter for video publish date. Options: today, this_week, this_month, this_year.
Name
is_active
Type
boolean
Description
Whether the configuration is active. Defaults to true. Set to false to create a paused configuration.

Request

POST

/comet

curl -X POST https://api.virlo.ai/comet \
  -H "Authorization: Bearer {token}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Tech Reviews",
    "keywords": ["tech review", "gadget review", "product review"],
    "platforms": ["youtube", "tiktok"],
    "cadence": "daily",
    "min_views": 10000,
    "time_range": "this_week",
    "is_active": true
  }'

Response

{
  "results": 1,
  "data": {
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "name": "Tech Reviews",
    "keywords": ["tech review", "gadget review", "product review"],
    "platforms": ["youtube", "tiktok"],
    "cadence": "0 0 * * *",
    "min_views": 10000,
    "time_range": "this_week",
    "is_active": true,
    "is_processing": false,
    "last_run_at": null,
    "next_run_at": "2024-01-16T00:00:00.000Z",
    "created_at": "2024-01-15T10:30:00.000Z",
    "updated_at": "2024-01-15T10:30:00.000Z"
  }
}

GET/comet/list

List comets

Retrieves a list of all your comet configurations. By default, only active configurations are returned. Use the include_inactive parameter to see all configurations including deactivated ones.

Cost per request:

Query parameters

Name
include_inactive
Type
string
Description
Include inactive configurations in the results. Set to "true" to include all configurations. Defaults to showing only active ones.

Request

GET

/comet/list

curl -G https://api.virlo.ai/comet/list \
  -H "Authorization: Bearer {token}" \
  -d include_inactive=true

Response

{
  "results": 2,
  "data": {
    "total": 2,
    "configurations": [
      {
        "id": "123e4567-e89b-12d3-a456-426614174000",
        "name": "Tech Reviews",
        "keywords": ["tech review", "gadget review"],
        "platforms": ["youtube", "tiktok"],
        "cadence": "0 0 * * *",
        "min_views": 10000,
        "time_range": "this_week",
        "is_active": true,
        "is_processing": false,
        "last_run_at": "2024-01-15T00:00:00.000Z",
        "next_run_at": "2024-01-16T00:00:00.000Z",
        "created_at": "2024-01-15T10:30:00.000Z",
        "updated_at": "2024-01-15T10:30:00.000Z"
      },
      {
        "id": "987f6543-e21b-34d5-c678-789012345678",
        "name": "Gaming Content",
        "keywords": ["gaming", "gameplay"],
        "platforms": ["youtube"],
        "cadence": "0 0 * * 0",
        "min_views": 50000,
        "time_range": "this_month",
        "is_active": false,
        "is_processing": false,
        "last_run_at": "2024-01-10T00:00:00.000Z",
        "next_run_at": null,
        "created_at": "2024-01-10T08:00:00.000Z",
        "updated_at": "2024-01-14T12:00:00.000Z"
      }
    ]
  }
}

GET/comet/:id

Get comet

Retrieves a specific comet configuration by its unique ID. Returns detailed information including the schedule, filtering criteria, and execution history.

Cost per request:

Path parameters

Name
id
Type
string
Description
UUID of the comet configuration. Example: 123e4567-e89b-12d3-a456-426614174000

Request

GET

/comet/:id

curl https://api.virlo.ai/comet/123e4567-e89b-12d3-a456-426614174000 \
  -H "Authorization: Bearer {token}"

Response

{
  "results": 1,
  "data": {
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "name": "Tech Reviews",
    "keywords": ["tech review", "gadget review"],
    "platforms": ["youtube", "tiktok"],
    "cadence": "0 0 * * *",
    "min_views": 10000,
    "time_range": "this_week",
    "is_active": true,
    "is_processing": false,
    "last_run_at": "2024-01-15T00:00:00.000Z",
    "next_run_at": "2024-01-16T00:00:00.000Z",
    "created_at": "2024-01-15T10:30:00.000Z",
    "updated_at": "2024-01-15T10:30:00.000Z"
  }
}

PUT/comet/:id

Update comet

Updates an existing comet configuration. All fields are optional - only provided fields will be updated. This allows you to modify scheduling, keywords, platforms, or filters without recreating the entire configuration.

Cost per request:

Path parameters

Name
id
Type
string
Description
UUID of the comet configuration to update. Example: 123e4567-e89b-12d3-a456-426614174000

Request body

All fields are optional. Only include the fields you want to update.

Name
name
Type
string
Description
Update the configuration name.
Name
keywords
Type
array
Description
Update the keywords array (1-20 keywords).
Name
platforms
Type
array
Description
Update the platforms to scrape from.
Name
cadence
Type
string
Description
Update the scheduling cadence.
Name
min_views
Type
number
Description
Update the minimum view count threshold.
Name
time_range
Type
string
Description
Update the time range filter.
Name
is_active
Type
boolean
Description
Activate or deactivate the configuration.

Request

PUT

/comet/:id

curl -X PUT https://api.virlo.ai/comet/123e4567-e89b-12d3-a456-426614174000 \
  -H "Authorization: Bearer {token}" \
  -H "Content-Type: application/json" \
  -d '{
    "min_views": 50000,
    "time_range": "this_month",
    "cadence": "weekly"
  }'

Response

{
  "results": 1,
  "data": {
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "name": "Tech Reviews",
    "keywords": ["tech review", "gadget review"],
    "platforms": ["youtube", "tiktok"],
    "cadence": "0 0 * * 0",
    "min_views": 50000,
    "time_range": "this_month",
    "is_active": true,
    "is_processing": false,
    "last_run_at": "2024-01-15T00:00:00.000Z",
    "next_run_at": "2024-01-21T00:00:00.000Z",
    "created_at": "2024-01-15T10:30:00.000Z",
    "updated_at": "2024-01-15T12:00:00.000Z"
  }
}

DELETE/comet/:id

Delete comet

Soft deletes (deactivates) a comet configuration by setting is_active to false. The configuration record remains in the database but will no longer be scheduled for scraping. You can reactivate it later using the update endpoint.

Cost per request:

Path parameters

Name
id
Type
string
Description
UUID of the comet configuration to deactivate. Example: 123e4567-e89b-12d3-a456-426614174000

Request

DELETE

/comet/:id

curl -X DELETE https://api.virlo.ai/comet/123e4567-e89b-12d3-a456-426614174000 \
  -H "Authorization: Bearer {token}"

Response

{
  "message": "Comet configuration deactivated successfully"
}

GET/comet/:id/videos

Get comet videos

Retrieves all videos that have been linked to a specific comet configuration. Supports comprehensive filtering, pagination, and sorting options to help you analyze your collected video data.

Cost per request:

Path parameters

Name
id
Type
string
Description
UUID of the comet configuration. Example: 123e4567-e89b-12d3-a456-426614174000

Query parameters

Name
minViews
Type
number
Description
Filter videos by minimum view count. Example: 10000
Name
platforms
Type
string
Description
Filter by platform(s). Can be a single platform or comma-separated string. Options: youtube, tiktok, instagram. Example: "youtube,tiktok"
Name
startDate
Type
string
Description
Filter videos published after this date (ISO date string). Example: "2024-01-01T00:00:00Z"
Name
endDate
Type
string
Description
Filter videos published before this date (ISO date string). Example: "2024-12-31T23:59:59Z"
Name
limit
Type
number
Description
Maximum number of videos to return (1-1000). Default is 50.
Name
page
Type
number
Description
Page number for pagination (starts at 1). Default is 1.
Name
orderBy
Type
string
Description
Field to sort by. Options: publish_date, views, created_at. Defaults to unsorted.
Name
orderDirection
Type
string
Description
Sort direction: asc or desc. Default is desc.

Request

GET

/comet/:id/videos

curl -G https://api.virlo.ai/comet/123e4567-e89b-12d3-a456-426614174000/videos \
  -H "Authorization: Bearer {token}" \
  -d minViews=50000 \
  -d platforms=youtube,tiktok \
  -d orderBy=views \
  -d orderDirection=desc \
  -d limit=20 \
  -d page=1

Response

{
  "results": 1,
  "data": {
    "custom_niche_id": "123e4567-e89b-12d3-a456-426614174000",
    "custom_niche_name": "Tech Reviews",
    "total": 150,
    "limit": 20,
    "offset": 0,
    "videos": [
      {
        "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
        "title": "Best Tech Review 2024",
        "platform": "youtube",
        "views": 125000,
        "publish_date": "2024-01-14T15:30:00.000Z"
      },
      {
        "url": "https://www.tiktok.com/@techguru/video/7318456789012345678",
        "title": "Gadget Unboxing",
        "platform": "tiktok",
        "views": 98000,
        "publish_date": "2024-01-14T10:20:00.000Z"
      }
    ]
  }
}

Usage workflow

The Comet custom niche system follows a straightforward workflow:

Create configuration: POST to /comet with your keywords (max 20), platforms, and scheduling preferences
Save the ID: Store the returned configuration id for future reference
Monitor execution: Use GET /comet/:id to check when the configuration last ran and when it will run next
List configurations: GET /comet/list to see all your custom niche configurations
Retrieve videos: GET /comet/:id/videos to access all collected videos with flexible filtering
Update as needed: PUT /comet/:id to modify keywords, scheduling, or filters
Pause when done: DELETE /comet/:id to deactivate (can be reactivated later with PUT)

The configuration runs automatically on your specified cadence (daily, weekly, monthly, or custom cron). Each execution collects videos matching your criteria and makes them available through the videos endpoint.

Scheduling options

Simple cadences

Use these convenient shortcuts for common scheduling patterns:

"daily" - Runs once per day at midnight UTC (equivalent to "0 0 * * *")
"weekly" - Runs once per week on Sunday at midnight UTC (equivalent to "0 0 * * 0")
"monthly" - Runs once per month on the 1st at midnight UTC (equivalent to "0 0 1 * *")

Cron expressions

For more control, use cron expressions with a minimum frequency of once per day:

┌───────────── minute (0 - 59)
│ ┌───────────── hour (0 - 23)
│ │ ┌───────────── day of month (1 - 31)
│ │ │ ┌───────────── month (1 - 12)
│ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday)
│ │ │ │ │
* * * * *

Examples:

"0 9 * * *" - Every day at 9:00 AM UTC
"0 0,12 * * *" - Twice daily at midnight and noon UTC
"0 0 * * 1,4" - Every Monday and Thursday at midnight UTC
"30 14 1 * *" - 1st of every month at 2:30 PM UTC

Note: Cadence expressions with a frequency higher than once per day (e.g., every hour) are not supported to prevent excessive API usage.

Best practices

Keyword selection

Use specific phrases: Multi-word keywords like "tech product review" yield better results than single words like "tech"
Avoid ambiguous terms: Generic words return scattered, unrelated content
Combine related keywords: Group variations like ["iPhone review", "iPhone unboxing", "iPhone tutorial"]
Stay within limits: Maximum 20 keywords per configuration
Test and refine: Start with a few keywords, check the results, then expand or adjust

Platform targeting

YouTube: Best for long-form content, tutorials, reviews
TikTok: Optimal for short-form viral content, trends
Instagram: Great for visual content, creator posts

Use all platforms for comprehensive coverage, or target specific platforms based on your content needs.

View thresholds

Low threshold (0-10k): Captures emerging content and smaller creators
Medium threshold (10k-100k): Filters for proven performers with solid engagement
High threshold (100k+): Focuses on viral hits and top-performing content

Adjust min_views based on your quality vs. quantity preference.

Time ranges

today: Most recent content, often still gaining traction
this_week: Balance of fresh content with established engagement
this_month: Proven performers with substantial data
this_year: Long-term trends and evergreen content

Match the time range to your analysis goals and update frequency.

Scheduling strategy

Daily: Best for fast-moving topics, news, trends
Weekly: Good for regular monitoring without overwhelming data
Monthly: Suitable for slow-moving niches or periodic reports
Custom cron: Advanced scheduling for specific times or days

Consider your analysis frequency and data processing capacity when choosing a cadence.

Use cases

Content monitoring

Track specific topics or themes over time:

{
  name: "AI Content Trends",
  keywords: ["ChatGPT", "AI tools", "artificial intelligence"],
  platforms: ["youtube", "tiktok"],
  cadence: "daily",
  min_views: 10000,
  time_range: "this_week"
}

Competitor analysis

Monitor competitors' content performance:

{
  name: "Competitor Tracking",
  keywords: ["competitor brand", "competitor product", "vs competitor"],
  platforms: ["youtube", "tiktok", "instagram"],
  cadence: "weekly",
  min_views: 50000,
  time_range: "this_month"
}

Niche discovery

Explore emerging content verticals:

{
  name: "Emerging Gaming Trends",
  keywords: ["new game", "indie game", "game release"],
  platforms: ["youtube"],
  cadence: "0 0 * * 1",  // Monday mornings
  min_views: 5000,
  time_range: "this_week"
}

Brand mentions

Track how your brand is discussed:

{
  name: "Brand Mentions",
  keywords: ["your brand", "your product", "your company"],
  platforms: ["youtube", "tiktok", "instagram"],
  cadence: "daily",
  min_views: 0,  // Catch all mentions
  time_range: "today"
}

Industry research

Analyze broad industry trends:

{
  name: "SaaS Industry Trends",
  keywords: ["SaaS tools", "B2B software", "productivity apps"],
  platforms: ["youtube"],
  cadence: "monthly",
  min_views: 100000,
  time_range: "this_month"
}

Error responses

The Comet endpoints may return the following error codes:

Name
400 Bad Request
Description
Invalid parameters provided. Common causes include:
- Missing required fields (name, keywords, platforms, cadence, min_views, time_range)
- Keywords array not between 1-50 elements
- Invalid platform values (must be 'youtube', 'tiktok', or 'instagram')
- Invalid time_range value
- Invalid cadence format
- min_views less than 0
- Invalid pagination parameters
Name
401 Unauthorized
Description
Missing or invalid API key. Ensure you're including your API key in the Authorization header.
Name
404 Not Found
Description
Comet configuration not found. The specified ID doesn't exist or you don't have access to it.
Name
429 Too Many Requests
Description
Rate limit exceeded. Your plan has specific rate limits per endpoint. Wait before making additional requests.