Skip to content

Log Management and Rotation

Purpose: Guide to automated log rotation for OraDBA system logs and Oracle Database logs using logrotate integration.

Audience: System administrators and DBAs managing log files.

Prerequisites:

  • logrotate installed (standard on most Linux distributions)
  • Root access for logrotate configuration
  • Understanding of log file locations

Overview

OraDBA provides comprehensive log management capabilities through logrotate integration. This chapter covers the configuration, deployment, and management of automated log rotation for both OraDBA system logs and Oracle Database logs.

Proper log management is critical for:

  • Disk Space Management: Prevent uncontrolled log growth
  • System Performance: Reduce I/O overhead from excessive logging
  • Compliance: Meet retention requirements (PCI-DSS, HIPAA, SOX, GDPR)
  • Security: Maintain audit trails
  • Troubleshooting: Keep relevant historical data

Quick Start

System-Wide Installation (Traditional)

# Install all logrotate templates (requires root)
sudo oradba_logrotate.sh --install

# Or install specific templates
sudo oradba_logrotate.sh --install --template oracle-alert

User-Mode Installation (Non-Root)

# Set up user-specific logrotate configurations
oradba_logrotate.sh --install-user

# Run manually
oradba_logrotate.sh --run-user

# Generate crontab entry for automation
oradba_logrotate.sh --cron

Testing

# Test configuration without rotating logs
oradba_logrotate.sh --test

# Force rotation for testing (requires root)
sudo oradba_logrotate.sh --force

Customization

# Generate customized configurations for your environment
oradba_logrotate.sh --customize

# Review generated configs in /tmp/logrotate-custom-*
ls -la /tmp/logrotate-custom-*/

Log Management Components

OraDBA provides five logrotate templates and a management script:

1. OraDBA System Logs (oradba.logrotate)

Manages logs generated by OraDBA itself:

  • Installation logs: /var/log/oradba/install/*.log (monthly, 12 months)
  • Operational logs: /var/log/oradba/ops/*.log (weekly, 8 weeks)
  • User logs: /var/log/oradba/user/*.log (weekly, 4 weeks)
  • Backup logs: /var/log/oradba/backup/*.log (weekly, 12 weeks)

2. Database Alert Logs (oracle-alert.logrotate)

Rotates Oracle alert logs without disrupting the database:

  • Pattern: $ORACLE_BASE/diag/rdbms/*/*/trace/alert_*.log
  • Method: copytruncate (safe for open files)
  • Schedule: Daily rotation, 30 days retention
  • Size trigger: 100MB maximum before forced rotation

3. Trace Files (oracle-trace.logrotate)

Cleans up diagnostic trace files:

  • Background traces: 30 days retention
  • Metadata files: 14 days retention
  • Core dumps: 7 days retention
  • RMAN traces: 30 days retention
  • Data Pump traces: 30 days retention

4. Audit Logs (oracle-audit.logrotate)

Manages audit trails with compliance considerations:

  • XML audit files: 90 days default (configurable for compliance)
  • Traditional audit: 90 days retention
  • Unified audit spillover: 90 days
  • Fine-Grained Audit: 90 days
  • Database Vault logs: 90 days

Compliance Requirements:

Standard Retention Period Notes
PCI-DSS 1 year minimum Card payment data
HIPAA 6 years minimum Healthcare data
SOX 7 years minimum Financial records
GDPR Varies by data EU privacy law

5. Listener Logs (oracle-listener.logrotate)

Rotates Oracle Net listener logs:

  • listener.log: Daily rotation, 30 days retention
  • Trace files: Weekly rotation, 30 days retention
  • Alert logs: Weekly rotation, 30 days retention
  • Incident dumps: Weekly rotation, 30 days retention

Management Script

oradba_logrotate.sh

Central management tool for logrotate configurations with support for both system-wide (root) and user-mode (non-root) operation.

Options

System-wide (requires root):

-i, --install             Install logrotate configurations to /etc/logrotate.d
-u, --uninstall          Remove system-wide configurations
-f, --force              Force rotation (for testing)

User-mode (non-root):

--install-user           Set up user-specific logrotate configurations
--run-user               Run logrotate with user-specific configurations
**System-Wide (Root) Mode:**

```bash
# Install all templates
sudo oradba_logrotate.sh --install

# Test all configurations
oradba_logrotate.sh --test

# Force rotation (testing only)
sudo oradba_logrotate.sh --force

# List installed configs
oradba_logrotate.sh --list

# Uninstall
sudo oradba_logrotate.sh --uninstall

User-Mode (Non-Root):

# Initial setup
oradba_logrotate.sh --install-user

# Test user-mode configuration
oradba_logrotate.sh --test

# Run rotation manually
oradba_logrotate.sh --run-user

# Generate crontab entry
oradba_logrotate.sh --cron

# Add to crontab
crontab -e  # Then paste the generated entry

Customization:

# Generate customized configs
oradba_logrotate.sh --customize-install --template oracle-alert

# Test all configurations
oradba_logrotate.sh --test

# Test specific configuration
oradba_logrotate.sh --test --template oracle-audit

# Generate customized configs
oradba_logrotate.sh --customize

# Force rotation (testing only)
sudo oradba_logrotate.sh --force --template oracle-alert

# List installed configs
oradba_logrotate.sh --list

Configuration Customization

User-Mode Operation

OraDBA supports non-root logrotate operation for environments where root access is restricted or per-user log management is preferred.

Benefits

  • No root access required: DBAs can manage logs without sudo
  • User-specific state: Each user has independent rotation tracking
  • Flexible scheduling: Run manually or via user's crontab
  • Isolated from system: Doesn't interfere with system-wide logrotate

Setup Process

  1. Initialize user-mode configuration:
oradba_logrotate.sh --install-user

This creates: - ~/.oradba/logrotate/ - Configuration directory - ~/.oradba/logrotate/state/ - State files directory - oracle-alert.logrotate - Alert log configuration - oracle-trace.logrotate - Trace files configuration - oracle-listener.logrotate - Listener log configuration

  1. Review and customize configurations:
ls -l ~/.oradba/logrotate/
vi ~/.oradba/logrotate/oracle-alert.logrotate
  1. Test the configuration:
oradba_logrotate.sh --test
  1. Run manually:
oradba_logrotate.sh --run-user
  1. Automate with cron:
# Generate crontab entry
oradba_logrotate.sh --cron

# Add to crontab
crontab -e
# Paste: 0 2 * * * /path/to/oradba_logrotate.sh --run-user >/dev/null 2>&1

User-Mode vs System-Wide

Aspect User-Mode System-Wide
Privileges No root required Requires root
Installation ~/.oradba/logrotate/ /etc/logrotate.d/
State files ~/.oradba/logrotate/state/ /var/lib/logrotate/
Execution Manual or user crontab System cron
Scope User's Oracle logs All system logs
Management Self-service System admin

User-Mode Limitations

  • Requires logrotate binary accessible in PATH
  • Cannot rotate logs owned by other users (unless permissions allow)
  • State tracking is per-user (different users track rotation independently)
  • Not integrated with system's logrotate.d scheduling

Environment-Specific Configuration

The --customize option generates configurations tailored to your environment:

oradba_logrotate.sh --customize

This creates:

  • /tmp/logrotate-custom-<timestamp>/ directory
  • Configurations with actual paths (replacing $ORACLE_BASE wildcards)
  • Per-SID configurations discovered from /etc/oratab
  • Ready-to-deploy files for review

Manual Customization

Edit installed configurations in /etc/logrotate.d/:

# Edit retention periods
sudo vi /etc/logrotate.d/oracle-alert

# Change from:
rotate 30  # Keep 30 days

# To:
rotate 365  # Keep 1 year for compliance

Common Customization Scenarios

Compliance Requirements

For PCI-DSS (1 year audit retention):

# Edit /etc/logrotate.d/oracle-audit
# Change rotate value
rotate 365  # Daily rotation = 365 days

For HIPAA (6 years):

# Use weekly rotation with long retention
weekly
rotate 312  # 6 years * 52 weeks

Disk Space Constraints

Reduce retention for tight storage:

# Edit alert log rotation
/path/to/alert_*.log {
    daily
    rotate 7  # Only keep 1 week
    maxsize 50M  # Rotate at 50MB instead of 100MB
    # ... rest of config
}

High-Volume Systems

For systems with heavy logging:

# More frequent rotation
/path/to/alert_*.log {
    hourly  # Rotate every hour
    rotate 168  # Keep 1 week (24*7)
    maxsize 10M  # Smaller rotation size
    # ... rest of config
}

Integration with Monitoring

Log Rotation Monitoring

Monitor logrotate execution:

# Check logrotate status
ls -lrt /var/lib/logrotate/status

# Review logrotate logs
grep logrotate /var/log/messages
grep logrotate /var/log/syslog

Alert on Rotation Failures

Example monitoring script:

#!/bin/bash
# check_logrotate.sh - Monitor logrotate status

LAST_RUN=$(stat -c %Y /var/lib/logrotate/status 2>/dev/null || echo 0)
CURRENT_TIME=$(date +%s)
AGE=$((CURRENT_TIME - LAST_RUN))

# Alert if no rotation in 36 hours
if [ "$AGE" -gt 129600 ]; then
    echo "WARNING: Logrotate hasn't run in $((AGE / 3600)) hours"
    exit 1
fi

echo "OK: Logrotate last run $((AGE / 3600)) hours ago"
exit 0

Log Growth Monitoring

Monitor log directory sizes:

#!/bin/bash
# monitor_log_growth.sh

ORACLE_BASE=${ORACLE_BASE:-/opt/oracle}
THRESHOLD_MB=5000  # 5GB warning threshold

for logdir in "$ORACLE_BASE"/diag/rdbms/*/*/trace; do
    if [ -d "$logdir" ]; then
        SIZE_MB=$(du -sm "$logdir" | awk '{print $1}')
        if [ "$SIZE_MB" -gt "$THRESHOLD_MB" ]; then
            echo "WARNING: $logdir is ${SIZE_MB}MB (threshold: ${THRESHOLD_MB}MB)"
        fi
    fi
done

Troubleshooting

Common Issues

Rotation Not Working

Symptom: Logs continue growing despite logrotate configuration

Diagnosis:

# Test configuration
oradba_logrotate.sh --test --template oracle-alert

# Check for errors
sudo logrotate -d /etc/logrotate.d/oracle-alert

Common Causes:

  1. Incorrect file paths or wildcards
  2. Permission issues (logrotate runs as root)
  3. SELinux denials
  4. Syntax errors in configuration

Permission Errors

Symptom: permission denied errors in logrotate output

Solution:

# Verify file ownership
ls -la $ORACLE_BASE/diag/rdbms/*/*/trace/

# Ensure logrotate can read/write
# Option 1: Use 'su oracle oracle' in config
# Option 2: Adjust permissions (not recommended)

Logs Not Compressed

Symptom: Rotated logs remain uncompressed

Diagnosis:

# Check compress directive exists
grep -A5 "compress" /etc/logrotate.d/oracle-alert

# Verify delaycompress is intentional

Solution: Check compress, nocompress, and delaycompress directives

Database Can't Write to Alert Log

Symptom: Database logging stops after rotation

Cause: Using create instead of copytruncate for active logs

Solution: Always use copytruncate for logs that remain open:

/path/to/alert_*.log {
    copytruncate  # REQUIRED for active logs
    # ... rest of config
}

Verification Steps

After installation or changes:

# 1. Test configuration syntax
oradba_logrotate.sh --test

# 2. Verify file paths match actual logs
ls -la $ORACLE_BASE/diag/rdbms/*/*/trace/alert_*.log

# 3. Check logrotate status file
cat /var/lib/logrotate/status | grep oracle

# 4. Force rotation to test (non-production only!)
sudo oradba_logrotate.sh --force --template oracle-alert

# 5. Verify rotated files exist
ls -la $ORACLE_BASE/diag/rdbms/*/*/trace/alert_*.log.*

Debug Mode

Run logrotate in debug mode:

# Debug specific config
sudo logrotate -d -v /etc/logrotate.d/oracle-alert

# Debug all configs
sudo logrotate -d -v /etc/logrotate.conf

Best Practices

1. Test Before Deployment

Always test configurations before production deployment:

# Generate customized configs
oradba_logrotate.sh --customize

# Review generated files
cat /tmp/logrotate-custom-*/oracle-alert

# Test syntax
logrotate -d /tmp/logrotate-custom-*/oracle-alert

# Deploy after validation
sudo cp /tmp/logrotate-custom-*/oracle-alert /etc/logrotate.d/

2. Backup Existing Configurations

Management script automatically backs up existing configs:

# Backups stored in:
/etc/logrotate.d/backups/<timestamp>/

Manual backup before changes:

sudo cp -a /etc/logrotate.d /etc/logrotate.d.backup.$(date +%Y%m%d)

3. Monitor Disk Space

Set up alerts for filesystem utilization:

# Example: Alert at 80% full
df -h $ORACLE_BASE | awk 'NR==2 {if (int($5) > 80) print "WARNING: "$6" is "$5" full"}'

4. Align with Backup Strategy

Coordinate log retention with backup policies:

# Example: Keep logs until after backup completes
# If weekly full backup, keep at least 8-10 days of logs

5. Document Compliance Requirements

Maintain documentation for audit purposes:

# Document in config comments
# PCI-DSS: Must retain 1 year
# Retention: 365 days (daily rotation)
# Review Date: 2025-12-22
# Approved By: DBA Team

6. Regular Reviews

Schedule periodic reviews:

  • Monthly: Verify rotation is working, check disk space trends
  • Quarterly: Review retention policies against compliance requirements
  • Annually: Audit full configuration, update for changes

Security Considerations

File Permissions

Ensure appropriate permissions:

# Logrotate configs should be root-owned
sudo chown root:root /etc/logrotate.d/oracle-*
sudo chmod 644 /etc/logrotate.d/oracle-*

# Log directories should be oracle-owned
chown -R oracle:oinstall $ORACLE_BASE/diag/

SELinux

If SELinux is enforcing, audit logs may be needed:

# Check for denials
sudo ausearch -m avc -ts recent | grep logrotate

# Create policy if needed
sudo audit2allow -a -M logrotate_oracle
sudo semodule -i logrotate_oracle.pp

Audit Trail Integrity

For compliance, consider:

  • Write-once storage: Archive audit logs to immutable storage
  • Checksums: Generate checksums before archiving
  • Encryption: Encrypt archived logs at rest
  • Access controls: Restrict who can modify logrotate configs

Template Reference

Available Templates

All templates located in src/templates/logrotate/:

Template Purpose Default Rotation Retention
oradba.logrotate OraDBA system logs Monthly/Weekly 12 months/8 weeks
oracle-alert.logrotate Database alert logs Daily 30 days
oracle-trace.logrotate Trace files cleanup Weekly (maxage) 30 days
oracle-audit.logrotate Audit logs Weekly 90 days
oracle-listener.logrotate Listener logs Daily 30 days

Template Features

Each template includes:

  • OraDBA headers: Version, author, description metadata
  • Inline documentation: Comments explaining each directive
  • Safe defaults: Conservative settings suitable for most environments
  • Compression: gzip compression with delayed compression where appropriate
  • Error handling: missingok, notifempty to prevent failures
  • Permissions: Appropriate create modes or copytruncate for active logs

Additional Resources

Logrotate Documentation

Oracle Documentation

  • Alert Log Management: Oracle Database Administrator's Guide
  • Diagnostic Framework (ADR): Oracle Database Utilities Guide
  • Audit Trail: Oracle Database Security Guide

OraDBA Documentation

Summary

OraDBA's log management system provides:

  • Five production-ready templates covering all Oracle and OraDBA logs
  • Management script for easy deployment and testing
  • Compliance support for PCI-DSS, HIPAA, SOX, GDPR
  • Customization options for environment-specific needs
  • Safe defaults with copytruncate for active logs

For most installations, the default templates work out-of-the-box. Use the management script to test and deploy, then customize retention periods based on your compliance requirements and disk space constraints.

Remember: Test configurations before production deployment, monitor rotation execution, and review policies regularly to ensure they meet your evolving needs.

Previous: SQLNet Configuration
Next: Usage Guide