URL Whitelisting
Witboost uses a URL Provider to fetch entity definitions from external sources. To enhance security and control which URLs can be processed, Witboost implements a URL whitelisting mechanism that restricts entities imports to only authorized sources.
Overview
The URL whitelisting feature allows administrators to define a list of allowed URL patterns that Witboost can process. This prevents unauthorized or potentially malicious entities imports by restricting the sources from which entity definitions can be fetched.
Configuration
URL whitelisting is configured through the locationsWhitelist
setting in your values.yaml
file:
# inside your values.yaml
ui:
appConfig:
# ... other configurations ...
catalog:
processingInterval: { minutes: 15 }
locationsWhitelist:
- ^https://github\.com/my-org/.*
- ^https://gitlab\.com/my-company/.*
- ^https://bitbucket\.org/my-team/.*
Pattern Types
The URL whitelist supports two types of patterns:
1. Regular Expression Patterns (Recommended)
Regular expressions provide the most flexible and powerful way to define URL patterns. They allow you to create precise matching rules for your allowed URLs.
Syntax: Use standard JavaScript regular expression syntax.
Examples:
locationsWhitelist:
# Match all GitHub repositories in a specific organization
- ^https://github\.com/my-organization/.*
# Match specific GitLab projects
- ^https://gitlab\.com/my-company/(project1|project2)/.*
# Match Bitbucket repositories with specific naming pattern
- ^https://bitbucket\.org/my-team/[a-z-]+/.*
# Match multiple domains
- ^https://(github\.com|gitlab\.com)/my-org/.*
# Match specific file types
- ^https://.*\.com/.*/catalog-info\.yaml$
2. Literal String Patterns (Fallback)
If a pattern is not a valid regular expression, the system treats it as a literal string and performs a simple substring match.
Examples:
locationsWhitelist:
# Simple domain matching
- github.com
- gitlab.com
# Specific path matching
- /catalog-info.yaml
Error Handling
When a URL is not whitelisted, the system:
- Logs a Warning: Records the blocked URL in the application logs
- Skips Processing: Does not attempt to fetch or process the URL
- Reports Error: Emits a processing error that can be viewed in the
Entities
page
Example Log Message:
URL https://unauthorized.com/catalog-info.yaml is not in the whitelist and will be skipped
Disabling Whitelisting
To disable URL whitelisting entirely (not recommended for production), set an empty whitelist:
# inside your values.yaml
ui:
appConfig:
# ... other configurations ...
catalog:
locationsWhitelist: []
Troubleshooting
Common Issues
- Pattern Not Matching: Ensure your regex is correct and properly escaped
- URLs Being Blocked: Check that your patterns are broad enough to match your URLs