Sensor Tasks
This section contains comprehensive guides for configuring sensor tasks in Datablast, including BigQuery sensors, cloud storage sensors, and custom sensor implementations.
Core Configuration
Section titled “Core Configuration”- Sensor Overview – Sensor task types and basic configuration
Sensor Types
Section titled “Sensor Types”- BigQuery Sensors – Table, partition, and query sensors
- Cloud Storage Sensors – GCS and S3 object sensors
- Custom Sensors – Custom sensor implementations
Specialized Configuration
Section titled “Specialized Configuration”- Sensor Parameters – Parameter configuration and validation
- Sensor Monitoring – Monitoring and alerting strategies
Key Features
Section titled “Key Features”Sensor Management
Section titled “Sensor Management”- Multiple Types: BigQuery, cloud storage, and custom sensors
- Parameter Configuration: Flexible parameter management
- Monitoring: Built-in monitoring and alerting
- Error Handling: Robust error management
Integration Support
Section titled “Integration Support”- BigQuery: Table, partition, and query-based sensors
- Cloud Storage: GCS and S3 object sensors
- Custom Logic: Custom sensor implementations
- Jinja Templates: Dynamic parameter templating
Quick Start
Section titled “Quick Start”- Choose Sensor Type: Select appropriate sensor for your use case
- Configure Parameters: Set up sensor parameters
- Set Monitoring: Configure monitoring and alerts
- Test Sensor: Validate sensor behavior
- Deploy: Deploy sensor to production