Skip to content

logowithtext

The pdpipe Python package provides a concise interface for building pandas pipelines that have pre-conditions, are verbose, support the fit-transform design of scikit-learn transformers and are highly serializable. pdpipe pipelines have a simple interface, informative prints and errors on pipeline application, support pipeline arithmetics and enable easier handling of mixed-type data.

>>> df = pd.DataFrame(
        data=[[4, 165, 'USA'], [2, 180, 'UK'], [2, 170, 'Greece']],
        index=['Dana', 'Jane', 'Nick'],
        columns=['Medals', 'Height', 'Born']
    )
>>> import pdpipe as pdp
>>> pipeline = pdp.ColDrop('Medals').OneHotEncode('Born')
>>> pipeline(df)
            Height  Born_UK  Born_USA
    Dana     165        0         1
    Jane     180        1         0
    Nick     170        0         0

Why pdpipe?

Ever written a preprocessing pipeline for pandas dataframes and had trouble serializing it for later deployment on a different machine? Ever needed fit-able preprocessing transformations, with tunable parameters that are inferred from training data, to be used later to transform input data? Ever struggled with preprocessing different types of data in the same pandas dataframe?

Enter pdpipe, a simple framework for serializable, chainable and verbose pandas pipelines. Its intuitive API enables you to generate, using only a few lines, complex pandas processing pipelines that can easily be broken down or composed together, examined and debugged, and that adhere to scikit-learn's Transformer API. Stop writing the same preprocessing boilerplate code again and again!

Installation

Install pdpipe using pip: pip install pdpipe

Getting Help

Getting Started

For a thorough overview of all the capabilities of pdpipe, continue to the "Getting Started".

Additionally, the awesome Tirthajyoti Sarkar wrote an excellent practical introduction on how to use pdpipe.


Last update: 2022-01-30
Back to top