DuckDB to join CSV and Parquet filesPython
Editor's note: although other snippets cover the ability to read CSV, JSON and parquet files, DuckDB's support for these files comes as table scanners. Table scanners act like internal DuckDB tables and you can JOIN and UNION the tables regardless of the original format.
Execute this Python
import duckdb conn = duckdb.connect() conn.execute(""" CREATE TABLE csv_table AS SELECT * FROM read_csv('path/to/your/csv/file.csv', header=True) """) conn.execute(""" CREATE TABLE parquet_table AS SELECT * FROM read_parquet('path/to/your/parquet/file.parquet') """) conn.execute(""" SELECT * FROM csv_table JOIN parquet_table ON csv_table.common_column = parquet_table.common_column """)
Copy code
Kin shah
Expand
Share link