USAJobs Historical API Data

Data collection last run: 2025-08-27

⚠️ This is not an official USAJobs project

Table of Contents

• Data Coverage by Year
• All Fields
• API Field Mapping & Normalization
• Code Examples from examples.py

Data Coverage by Year

All Fields

Note: This table shows fields from historical jobs data. Current jobs API contains additional fields that are preserved in the original nested structure but not included in this rationalized view.

API Field Mapping & Normalization

This dataset combines data from two USAJobs APIs with field rationalization for consistent querying:

API Sources

• Historical API (/api/historicjoa): Past job announcements by date range
• Current API (/api/Search): Currently active job postings
• API Documentation: https://developer.usajobs.gov/

Note: There is overlap between the APIs (2024-2025 jobs appear in both), but we collect from both APIs for completeness.

Important: The current_jobs_*.parquet files contain cumulative data - they include all jobs that have appeared in the Current API since we started collecting, not just jobs that are currently active. Once a job is added to these files, it remains there even after the position closes or is removed from the Current API. This provides a historical record of all jobs that were once "current."

Normalized Fields

Data Processing Approach

• Field Rationalization: Current API fields mapped to historical naming conventions for consistent querying
• Data Preservation: All original fields from both APIs retained alongside rationalized overlay fields
• No Data Loss: "Keep everything + overlay" approach ensures complete data accessibility
• Deduplication Available: Use usajobsControlNumber to identify records appearing in both APIs when needed

Data Structure

• Historical API: ~40 fields including nested arrays (HiringPaths, JobCategories, PositionLocations)
• Current API: ~25 overlay fields + full original nested structure (MatchedObjectDescriptor, etc.)
• Result: Query any year using consistent field names (e.g., hiringAgencyName, positionTitle) while retaining access to original nested structures

Code Examples from examples.py

The examples.py script demonstrates both local file access and direct GitHub downloads. It runs complete analysis on 2.97M job postings and automatically cleans up downloaded files.

Local Data Loading

import pandas as pd

# Load a single year
df_2024 = pd.read_parquet('data/historical_jobs_2024.parquet')
print(f"✓ Successfully loaded {len(df_2024):,} job postings from 2024")

GitHub Download Example

import pandas as pd
import requests

# Download directly from GitHub
url = 'https://github.com/abigailhaddad/usajobs_historical/raw/main/data/historical_jobs_2024.parquet'
df = pd.read_parquet(url)
print(f"✓ Downloaded {len(df):,} job postings")

Top Hiring Agencies Analysis

# 1. TOP 15 HIRING AGENCIES (2024 DATA)
top_agencies = df_2024['hiringAgencyName'].value_counts().head(15)
for i, (agency, count) in enumerate(top_agencies.items(), 1):
    percentage = count / len(df_2024) * 100
    print(f"{i:2d}. {agency}: {count:,} jobs ({percentage:.1f}%)")

Salary Analysis

# Salary statistics
records_with_salary = df_2024[df_2024['maximumSalary'].notna()]
print(f"Records with salary data: {len(records_with_salary):,} ({len(records_with_salary)/len(df_2024)*100:.1f}%)")

# Salary ranges
min_range = f"${records_with_salary['minimumSalary'].min():,.0f} - ${records_with_salary['minimumSalary'].max():,.0f}"
max_range = f"${records_with_salary['maximumSalary'].min():,.0f} - ${records_with_salary['maximumSalary'].max():,.0f}"
median_range = f"${records_with_salary['minimumSalary'].median():,.0f} - ${records_with_salary['maximumSalary'].median():,.0f}"

print(f"Minimum salary range: {min_range}")
print(f"Maximum salary range: {max_range}")  
print(f"Median salary range: {median_range}")

Multi-Year Analysis with DuckDB

import duckdb

# Create DuckDB connection and register multiple Parquet files
conn = duckdb.connect('download/usajobs.duckdb')

# Register Parquet files as external tables
years = ['2022', '2023', '2024']
for year in years:
    file_path = f'data/historical_jobs_{year}.parquet'
    conn.execute(f"CREATE OR REPLACE VIEW jobs_{year} AS SELECT * FROM '{file_path}'")

# Create unified view
conn.execute("""
    CREATE OR REPLACE VIEW all_jobs AS 
    SELECT * FROM jobs_2024 
    UNION ALL SELECT * FROM jobs_2023 
    UNION ALL SELECT * FROM jobs_2022
""")

# Query salary trends by year
salary_trends = conn.execute("""
    SELECT 
        EXTRACT(year FROM positionOpenDate) as year,
        ROUND(AVG(minimumSalary)) as avg_min,
        ROUND(AVG(maximumSalary)) as avg_max,
        ROUND(percentile_cont(0.5) WITHIN GROUP (ORDER BY minimumSalary)) as median_min,
        ROUND(percentile_cont(0.5) WITHIN GROUP (ORDER BY maximumSalary)) as median_max,
        COUNT(*) as jobs
    FROM all_jobs 
    WHERE minimumSalary IS NOT NULL 
    GROUP BY EXTRACT(year FROM positionOpenDate)
    ORDER BY year DESC
""").fetchdf()

print(salary_trends)

Monthly Hiring Patterns

# Monthly posting patterns
monthly_posts = df_2024.groupby(df_2024['positionOpenDate'].dt.month).size()
print("Monthly posting patterns:")
month_names = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']
for month, count in monthly_posts.items():
    percentage = count / len(df_2024) * 100
    print(f"    {month_names[month-1]}: {count:,} jobs ({percentage:.1f}%)")

Most Common Position Titles

# Most common position titles
common_titles = df_2024['positionTitle'].value_counts().head(10)
print("Most common position titles:")
for i, (title, count) in enumerate(common_titles.items(), 1):
    percentage = count / len(df_2024) * 100
    print(f"   {i:2d}. {title}: {count:,} ({percentage:.1f}%)")