{"dataset":{"id":"00000000-0000-0000-0000-000000000060","slug":"pandas-best-practices","name":"pandas best practices","description":"Source-linked guidance on pandas fundamentals: dataframes and indexing, groupby and aggregation, merging and joining, missing data, and performance and dtypes. Scoped to current pandas docs.","visibility":"public","version":1,"documentCount":5,"publisher":"Koskamo editorial","license":"Original summaries; linked sources retain their own terms.","reviewedAt":"2026-10-04T00:00:00.000Z","readOnly":true,"createdAt":"2026-10-04T00:00:00.000Z","updatedAt":"2026-10-04T00:00:00.000Z"},"total":5,"documents":[{"title":"Enhancing performance","sourceUrl":"https://pandas.pydata.org/docs/user_guide/enhancingperf.html","text":"For computationally heavy applications, pandas users may achieve sizable speed-ups by offloading work to Cython, with Numba and pandas.eval() also available; Cython and Numba generally offer a larger speedup than pandas.eval() but require a lot more code. Users are highly encouraged to install the recommended dependencies for pandas, which are often not installed by default but offer speed improvements if present. Before employing advanced techniques, refactor as much as possible in Python, for example by removing for-loops and making use of NumPy vectorization, as it is always worth optimising in Python first. The tutorial walks through cythonizing a slow computation applied row-wise to a DataFrame; the final cythonized solution is around 100 times faster than the pure Python solution. Scope: pandas. Reviewed October 4, 2026.","id":"pandas-best-practices:Enhancing performance","version":1,"createdAt":"2026-10-04T00:00:00.000Z","updatedAt":"2026-10-04T00:00:00.000Z"},{"title":"Working with missing data","sourceUrl":"https://pandas.pydata.org/docs/user_guide/missing_data.html","text":"Use numpy.nan for NumPy types (coerced to float64 or object), NaT for datetime64, timedelta64, and PeriodDtype, and NA for StringDtype, Int64Dtype, Float64Dtype, BooleanDtype, and ArrowDtype, preserving the original type. isna() and notna() detect missing values, treating None as missing. Equality comparisons between np.nan, NaT, and NA do not act like None. Cumulative methods like cumsum() ignore NA values by default but preserve them; use skipna=False to include NA values. dropna() drops rows or columns with missing data. fillna() replaces NA values with non-NA data; when the dtype is not object, fill values are replaced with the proper NA value for the dtype. ffill() and bfill() fill gaps forward or backward, optionally limited via limit. Products treat NA values or empty data as 1. Scope: pandas. Reviewed October 4, 2026.","id":"pandas-best-practices:Working with missing data","version":1,"createdAt":"2026-10-04T00:00:00.000Z","updatedAt":"2026-10-04T00:00:00.000Z"},{"title":"Merge, join, concatenate and compare","sourceUrl":"https://pandas.pydata.org/docs/user_guide/merging.html","text":"Use `concat()` to merge multiple Series or DataFrame objects along a shared axis, accepting a list or dict of homogeneously-typed inputs while optionally applying set logic (union or intersection) to indexes on other axes. For column-based combinations, employ `DataFrame.join()`. To update missing values with existing non-missing counterparts at identical locations, utilize `DataFrame.combine_first()`. Combine two objects using SQL-style logic via `merge()`, align along an ordered axis with `merge_ordered()`, or match by proximity rather than exact keys using `merge_asof()`. Finally, identify value discrepancies between two Series or DataFrames by calling `Series.compare()` or `DataFrame.compare()`. Scope: pandas. Reviewed October 4, 2026.","id":"pandas-best-practices:Merge, join, concatenate and compare","version":1,"createdAt":"2026-10-04T00:00:00.000Z","updatedAt":"2026-10-04T00:00:00.000Z"},{"title":"Group by: split-apply-combine","sourceUrl":"https://pandas.pydata.org/docs/user_guide/groupby.html","text":"Group by involves splitting data, applying functions, and combining results. The split step creates groups based on criteria, while the apply step performs aggregation (e.g., sums, means, counts), transformation (e.g., standardization, filling NAs), or filtration. Built-in GroupBy operations are more efficient than the apply method with user-defined Python functions. For complex or combined operations not fitting standard categories, GroupBy’s apply method examines results to sensibly combine them. GroupBy objects support mapping labels to group names via Python functions, lists, NumPy arrays, or dicts. Operations resemble those in the aggregating API, window API, and resample API. pandas aims to make SQL-like grouping natural. For advanced strategies, consult the cookbook. Scope: pandas. Reviewed October 4, 2026.","id":"pandas-best-practices:Group by: split-apply-combine","version":1,"createdAt":"2026-10-04T00:00:00.000Z","updatedAt":"2026-10-04T00:00:00.000Z"},{"title":"Indexing and selecting data","sourceUrl":"https://pandas.pydata.org/docs/user_guide/indexing.html","text":"Use .loc for label-based indexing and .iloc for integer position-based access, as direct operator usage limits optimization in production code. .loc raises KeyError for missing items, supporting labels, label slices (inclusive endpoints), boolean arrays, callables, and tuples. Treat numeric labels like 5 strictly as index labels, not positions. .iloc raises IndexError for out-of-bounds positions but permits out-of-bounds slices. It accepts integers, integer lists, integer slices, boolean arrays, callables, and tuples. Both .loc and .iloc accept callable indexers. Note that destructuring tuple keys into row and column indexes happens before callable application, preventing callables from returning tuples for multi-axis indexing. Omitted axes default to the null slice; for example, p.loc['a'] is equivalent to p.loc['a', :]. For advanced strategies, consult MultiIndex and cookbook documentation. Scope: pandas. Reviewed October 4, 2026.","id":"pandas-best-practices:Indexing and selecting data","version":1,"createdAt":"2026-10-04T00:00:00.000Z","updatedAt":"2026-10-04T00:00:00.000Z"}]}