Efficiently Manipulating Time in Pandas DataFrames with Timedelta - Adventures in Machine Learning (2024)

Adding and Subtracting Time in Pandas

In today’s article, we will learn how to add and subtract time in Pandas, a popular data analysis tool in Python. Time manipulation is a common task in data analysis and can reveal important insights into data trends and patterns.

Pandas provides multiple functionalities to perform time-related operations, including the Timedelta function. We will explore how to implement Timedelta to add and subtract time in a Pandas DataFrame.

Using Timedelta to Add Time

Adding time to a Pandas DataFrame can be done quickly with the Timedelta function. Timedelta is a fundamental time helper function in Python that can perform arithmetic on time values.

Let’s start by importing the Pandas library.

import pandas as pd

Next, let’s create a simple Pandas DataFrame to demonstrate how to use Timedelta to add time to a column in the DataFrame.

df = pd.DataFrame({'sales': [100, 200, 300, 400], 'dates': pd.to_datetime(['2022-01-01', '2022-01-02', '2022-01-03', '2022-01-04'])})

We have created a DataFrame with two columns, ‘sales’ and ‘dates.’ The ‘sales’ column contains integers, and the ‘dates’ column stores datetime values.

Now let’s use Timedelta to add 2 days to the ‘dates’ column in df using the “+” operator.

df['dates'] = df['dates'] + pd.Timedelta(days=2)

In the code above, we added 2 days to the ‘dates’ column in df by calling the Timedelta function with the keyword argument ‘days=2.’ We then use the “+” operator to add this Timedelta object to the ‘dates’ column.

To check whether the new dates were added correctly, we can use the “print” statement to display df.

print(df)

The output should be:

 sales dates0 100 2022-01-031 200 2022-01-042 300 2022-01-053 400 2022-01-06

As we can see above, the ‘dates’ column has been shifted 2 days into the future, while the ‘sales’ column has remained unchanged.

Using Timedelta to Subtract Time

The same logic applies when subtracting time from a Pandas DataFrame using Timedelta. Let’s use the same DataFrame created earlier and subtract 2 days from the ‘dates’ column.

df['dates'] = df['dates'] - pd.Timedelta(days=2)

The “-” operator is used in the code above to subtract the Timedelta object from the ‘dates’ column in df. We then print the DataFrame to check the result.

print(df)

The output will be:

 sales dates0 100 2019-12-301 200 2019-12-312 300 2020-01-013 400 2020-01-02

As we can see above, the ‘dates’ column has been shifted 2 days into the past, while the ‘sales’ column has remained unchanged. Example: Adding and Subtracting Time in a Pandas DataFrame

Now that we have learned how to use Timedelta to add and subtract time, let’s look at a practical example of how we can use it in a Pandas DataFrame to analyze and visualize data trends.

Creating a Pandas DataFrame

Let’s start by creating a simple Pandas DataFrame to demonstrate how to use Timedelta to add and subtract time in a column.

df = pd.DataFrame({ 'sales': [100, 200, 300, 400], 'date_added': pd.to_datetime(['2022-01-01', '2022-01-05', '2022-01-09', '2022-01-13'])})

The DataFrame contains two columns, ‘sales’ and ‘date_added’.

The ‘date_added’ column stores datetime values representing when the ‘sales’ data was added to the DataFrame.

Adding Time to a Column in a DataFrame

Now let’s add 3 months to the ‘date_added’ column using Timedelta to simulate how the data has changed over time.

df['date_added'] = df['date_added'] + pd.Timedelta(weeks=12)

In the code above, we added 3 months (12 weeks) to the ‘date_added’ column in df.

We then use the “+” operator to add this Timedelta object to the ‘date_added’ column. To check whether the new dates were added correctly, we can use the “print” statement to display df.

print(df)

The output should be:

 sales date_added0 100 2022-03-261 200 2022-04-232 300 2022-05-213 400 2022-06-18

As we can see above, the ‘date_added’ column has been shifted 3 months into the future while the ‘sales’ column has remained unchanged.

Subtracting Time from a Column in a DataFrame

Similarly, we can subtract a certain amount of time from a column in the DataFrame to analyze the data better. Let’s say that we want to view the sales data added in Q1 (January, February, and March).

q1_sales = df.loc[(df['date_added'] >= '2022-01-01') & (df['date_added'] <= '2022-03-31')]

In the code above, we created a new DataFrame ‘q1_sales’ by using the ‘loc’ function to slice out data that was added between January 1st and March 31st. We then use the “print” statement to display ‘q1_sales.’

print(q1_sales)

The output should be:

 sales date_added0 100 2022-03-26

As we can see above, only one row in the DataFrame satisfies the condition set to slice out Q1 data. However, by using Timedelta and subtracting dates, we can obtain more sales data for analysis.

q1_sales = df.loc[(df['date_added'] - pd.Timedelta(weeks=12) >= pd.to_datetime('2022-01-01')) & (df['date_added'] - pd.Timedelta(weeks=12) <= pd.to_datetime('2022-03-31'))]

In the code above, we first subtracted 3 months (12 weeks) from the ‘date_added’ column using Timedelta. We then used the ‘loc’ function to slice out data that was added between January 1st and March 31st from the modified ‘date_added’ column.

We then use the “print” statement to display ‘q1_sales.’

print(q1_sales)

The output should be:

 sales date_added0 100 2022-03-261 200 2022-04-23

As we can see above, by using Timedelta to subtract dates, we managed to obtain two rows of sales data for analysis in Q1.

Conclusion

In conclusion, Time manipulation is a common task in data analysis and can reveal important insights into data trends and patterns. Pandas provides multiple functionalities to perform time-related operations, including the Timedelta function.

We have learned how to use Timedelta to add and subtract time in a Pandas DataFrame. We have also worked through a practical example of how to use Timedelta to analyze and visualize data better.

By using Timedelta, we can quickly and efficiently manipulate time in Pandas, saving a lot of time and effort. In this article, we explored adding and subtracting time in Pandas, a prevalent data analysis tool in Python.

We learned how to use Timedelta to add and subtract time in a Pandas DataFrame. We also worked through a practical example of how to use Timedelta to analyze and visualize data better.

Timedelta is a powerful tool for time manipulation, which is a critical task in data analysis that can reveal important insights into data trends and patterns. By using Timedelta, we can quickly and efficiently manipulate time in Pandas, saving a lot of time and effort.

The takeaway is that Timedelta is an essential tool to have in your data analysis toolkit and can help you uncover valuable insights in your data.