⏰ Scheduler & Data Quality Checks

Tower includes a built-in scheduler that automatically refreshes your dashboard data and runs comprehensive data quality checks on every refresh. Get notified immediately if issues are detected, ensuring your analytics are always accurate and reliable.

🎯 What is the Scheduler?

Tower’s scheduler provides:

Automatic data refreshes on your chosen schedule
Data quality validation runs on every refresh
Email notifications when issues are detected
Configurable recipients for different dashboards
Detailed error reporting for troubleshooting
No manual intervention required

🚀 Setting Up Scheduled Refreshes

Accessing Scheduler Settings

Open your dashboard
Click the Settings or Configure icon
Navigate to the Scheduler section
Enable automatic refreshes

Configuring Refresh Schedule

Choose your refresh frequency:

Hourly - Every hour, on the hour
Daily - Specific time each day
Weekly - Specific day and time each week
Monthly - Specific date and time each month
Custom intervals - Every X hours/days

💡 Tip: Match your refresh schedule to your data source update frequency. If your database updates nightly, schedule refreshes for early morning.

Setting Refresh Times

For Daily Refreshes:

Choose the time in your local timezone
Consider off-peak hours for large datasets
Allow time before business hours for review

For Weekly Refreshes:

Select day of week (e.g., Monday, Friday)
Choose specific time
Ideal for weekly reporting cycles

For Monthly Refreshes:

Pick date (1-28 to avoid month-end issues)
Set specific time
Perfect for monthly executive reports

✅ Data Quality Checks

Automatic Validation

On every refresh, Tower automatically checks:

1. Schema Validation

Column names haven’t changed
Data types remain consistent
Required fields are present
No unexpected columns added/removed

2. Data Integrity

No null values in critical fields
Date ranges are valid
Numeric values within expected ranges
Referential integrity maintained

3. Volume Checks

Record counts within normal thresholds
No suspicious data spikes or drops
Comparison to historical patterns
Detection of empty or incomplete loads

4. Format Validation

Date formats consistent
Currency values properly formatted
Text fields within length limits
Special characters handled correctly

5. Business Logic Validation

Calculated fields compute correctly
Aggregations match expected values
Filters return appropriate results
Metrics align with business rules

📧 Email Notifications

Notification Configuration

Set up email alerts for:

✅ Data quality check failures
✅ Refresh completion (optional)
✅ Connection errors
✅ Schema change detections
✅ Data anomalies

Configuring Recipients

Add email addresses for:

Dashboard owner (automatic)
Team members who need to know
Data stewards responsible for data quality
Business users who rely on the dashboard
IT/Operations for technical issues

Add multiple recipients:

Go to Scheduler settings
Find “Notification Recipients” section
Enter email addresses (comma-separated)
Save changes

Notification Content

Email notifications include:

Dashboard name and link
Refresh timestamp
Issue description (if applicable)
Specific errors encountered
Affected data sources
Recommended actions
Historical context (how often this occurs)

🔍 Understanding Quality Check Results

✅ Successful Refresh

When everything passes:

Dashboard updates with latest data
No email sent (unless “success notifications” enabled)
Green status indicator in dashboard
Timestamp shows last successful refresh

⚠️ Warning Status

Minor issues detected:

Dashboard still updates
Warning email sent to recipients
Yellow status indicator
Issues listed in notification
Data may still be usable

Common warnings:

Slight volume fluctuation
Minor data format inconsistencies
Non-critical null values
Performance slower than usual

❌ Failed Refresh

Critical issues prevent update:

Dashboard shows previous data
Alert email sent immediately
Red status indicator
Detailed error information provided
Dashboard remains on last good data

Common failures:

Database connection timeout
Schema changes breaking queries
Authentication expired
Data source unavailable
Critical null values

🛠️ Troubleshooting Refresh Issues

Connection Problems

If you receive connection errors:

Verify database/API is online
Check credentials haven’t expired
Confirm firewall rules allow Tower access
Test connection in Data Sources page
Contact IT if network issues suspected

Schema Change Errors

If schema validation fails:

Review what changed in source system
Update data pipeline to accommodate changes
Remap affected fields
Test manually before re-enabling scheduler
Update semantic layer if needed

Performance Issues

If refreshes are slow or timing out:

Review dataset size and growth
Optimize queries in pipeline
Consider incremental refreshes
Add indexes to source database
Adjust timeout settings
Contact support for optimization help

💡 Best Practices

Scheduling Strategy

✅ Do:

Schedule during off-peak hours
Allow buffer time before business hours
Coordinate with source system updates
Test schedule changes during off-hours
Monitor refresh duration trends

❌ Don’t:

Schedule multiple large dashboards simultaneously
Refresh more frequently than data updates
Set schedules during peak business hours
Ignore repeated quality check warnings
Forget to update schedules when needs change

Notification Management

Set appropriate recipients:

Technical issues → IT/Data team
Data quality issues → Data stewards
Business impact → Dashboard users
Critical dashboards → Multiple contacts
Test dashboards → Minimal notifications

Monitoring & Maintenance

Regular reviews:

Check notification emails for patterns
Review refresh success rates monthly
Adjust schedules based on usage patterns
Archive/disable unused dashboards
Update quality check thresholds as needed

📊 Scheduler Dashboard

Viewing Refresh History

Access refresh logs to see:

Recent refresh attempts (success/failure)
Execution duration trends
Data quality check results over time
Error frequency patterns
Historical data volumes

Metrics to Monitor

Key scheduler metrics:

Success rate (aim for >95%)
Average refresh duration
Peak execution times
Error types and frequency
Data volume trends

Data Sources - Configure data connections
Data Pipelines - Set up transformations
Dashboards - Create and manage dashboards
Account Settings - Notification preferences

🎯 Advanced Features

Conditional Scheduling

Enterprise plan feature:

Schedule based on source data availability
Chain multiple dashboard refreshes
Conditional refresh based on data volume
Priority scheduling for critical dashboards

Custom Quality Rules

Define your own checks:

Business-specific validation rules
Custom threshold alerts
Industry-specific compliance checks
Multi-source consistency validation

Integration with Actions

Automate responses to quality issues:

Create JIRA tickets for data quality failures
Send Slack notifications to data team
Trigger alerts in monitoring systems
Update status in project management tools

❓ FAQ

Q: Can I manually trigger a refresh outside the schedule?
A: Yes! Click the refresh icon in your dashboard header to run an immediate refresh with quality checks.

Q: What happens if a refresh fails?
A: Your dashboard keeps showing the last successfully refreshed data, and you’ll receive an email with error details.

Q: Can different dashboards have different schedules?
A: Absolutely! Each dashboard has independent scheduler configuration.

Q: Do quality checks slow down refreshes?
A: Minimally. Quality checks run in parallel and typically add only a few seconds to refresh time.

Q: Can I disable quality checks?
A: Quality checks are always enabled to ensure data reliability. You can adjust notification thresholds.

Q: What if I want notifications via Slack instead of email?
A: Enterprise plans can configure custom notification channels. Contact support for setup.

Q: How long is refresh history retained?
A: Refresh logs are retained for 90 days on Professional plans, 1 year on Enterprise plans.

Q: Can I schedule refreshes based on source system updates?
A: Yes, on Enterprise plans you can configure event-based triggers instead of time-based schedules.

🆘 Need Help?

For scheduler issues:

Check notification emails for specific error details
Review refresh history in dashboard settings
Test data source connections manually
Verify credentials haven’t expired
Contact support@codygon.com with:
- Dashboard name
- Error message from notification
- Recent refresh history
- Data source details

📈 Scheduler Tips

For Daily Operations

Review morning notification emails with coffee
Set up filters to prioritize critical failures
Create dashboard for monitoring all refresh statuses
Document common issues and resolutions

For Data Teams

Use quality check logs to improve source data
Trend analysis on data volume and quality
Proactive monitoring of schema changes
Coordinate with source system maintenance schedules

For Executives

Configure digest emails for important dashboards only
Focus on critical quality issues
Delegate technical troubleshooting
Review scheduler health metrics quarterly