Data Processing Flow with Timestamp Addition

This code uses the Prefect library to create a data processing flow. It first generates random data, then processes these data by adding a timestamp, and finally runs the flow and prints the result.

Technology Stack : Prefect, NumPy, datetime

Code Type : The type of code

Code Difficulty : Intermediate

                
                    
import random
from prefect import task, flow
from datetime import datetime
import numpy as np

def generate_random_data(num_records):
    """
    Generate random data for demonstration purposes.
    """
    data = np.random.rand(num_records, 5)
    return data

@task
def process_data(data):
    """
    Process the data by adding a timestamp and returning it.
    """
    processed_data = np.append(data, np.array([datetime.now()]), axis=1)
    return processed_data

@flow
def data_processing_flow():
    data = generate_random_data(10)
    processed_data = process_data(data)
    return processed_data

# Running the flow
result = data_processing_flow()

print(result)