DuckDB to join CSV and Parquet filesPython

Editor's note: although other snippets cover the ability to read CSV, JSON and parquet files, DuckDB's support for these files comes as table scanners. Table scanners act like internal DuckDB tables and you can JOIN and UNION the tables regardless of the original format.

Execute this Python

import duckdb

conn = duckdb.connect()
conn.execute("""
CREATE TABLE csv_table AS
SELECT *
FROM read_csv('path/to/your/csv/file.csv', header=True)
""")

conn.execute("""
CREATE TABLE parquet_table AS
SELECT *
FROM read_parquet('path/to/your/parquet/file.parquet')
""")

conn.execute("""
SELECT *
FROM csv_table
JOIN parquet_table
ON csv_table.common_column = parquet_table.common_column
""")
Copy code

Kin shah

Edited 02/23/24