Top PySpark Coding Questions for Data Engineering Roles

Practice loading a CSV file and apply basic transformations such as selecting, filtering, and dropping columns.

Clean the dataset by filtering out or replacing null values in various columns.

Group data by customer and compute the total purchase amount per user.

Add a new column calculating discounted prices for products using arithmetic operations.

Read a nested JSON file and flatten it using explode and array-handling techniques.

Use window functions to find employees whose salary is higher than the department average.

Identify and remove duplicate records based on custom logic using window functions.

Implement a word count logic using PySpark RDD transformations on a text file.

Group records and aggregate values into lists using advanced group and array functions.

Summarize transactions month-wise by grouping and using date functions to extract months.

Generate a summary of top players using joins, aggregations, and string operations.

Calculate total sales for each store on a daily basis using grouping and aggregation.

Find the top 5 products with the highest total sales across all stores for a given day.

Given two years of product sales data, identify products whose total sales revenue has increased every year.

Given a dataset of trip costs and customer ratings, remove rows that contain outliers.