Choose a structure based on access pattern: lists for ordered items, dictionaries for key lookups, sets for membership checks, and tuples for fixed small records. This choice has direct impact on readability and performance.
- Map identifiers to records with dictionaries for fast retrieval.
- Use sets to remove duplicates and test membership efficiently.
- Keep nested structures explicit and documented when building pipelines.
Model operational data with dicts and lists
A common pattern is a list of dictionaries where each dictionary is one row-like record. This mirrors CSV or JSON payloads and keeps transformations straightforward before moving into pandas.
For repeated lookup by id, convert records to a dictionary keyed by that id. This avoids repeated list scans and clarifies intent.
Use sets and tuples intentionally
Sets are useful for uniqueness checks such as finding duplicate order IDs. Tuples work well for immutable coordinate-like values such as (year, month) keys in summary tables.
- Prefer explicit conversion instead of implicit structure changes.
- Name tuple elements through comments or dataclasses when meaning is not obvious.
- Keep dictionary keys consistent in type and format.
Detect duplicate order IDs quickly
A list of order records is loaded from two daily CSV files that may overlap.
- Build a list of order_id values from all records.
- Create a set from the same order_id list and compare lengths.
- If lengths differ, count repeated IDs and print the duplicates for review.
Common mistakes
- Using a list for frequent key-based lookups.
- Storing mixed key types such as int and str for the same identifier field.
- Assuming dictionary iteration order solves sorting requirements.
- Over-nesting structures until downstream code becomes hard to read.
Try one
Which structure is best when you need to check whether an order ID has been seen before many times?
A set, because membership checks are fast and the values are unique.
Sources
- Python documentationOfficial reference for Python syntax, standard library modules, and tutorials.