Project Gallery
A closer look at the dashboard in production and the code & architecture behind four of the pipelines on the Projects page.
Featured Dashboard
A Looker Studio report fed entirely by an automated GCP + Python pipeline — no manual refresh, no copy-pasted exports.
Crimes in India BI Report
Built on NCRB data, broken down by region and age group, refreshed automatically through a GCP pipeline rather than a manual export.
- Sources directly from BigQuery
- State & age-group breakdowns
- Geo map of regional crime density
Code Deep Dives: Architecture & Logic
Illustrative snippets reconstructing the core logic behind four pipelines from the Projects page — simplified for readability, since the originals are proprietary to their respective employers.
AI/LLM PII Redaction Logic
Hybrid rule-based + LLM verification pass used to scrub PII before content ships (Project 1).
Redaction Pass
# pii_redaction.py — hybrid rule + LLM redaction pass import re from langchain.chat_models import ChatOpenAI RULES = [ (re.compile(r"\b\d{10}\b"), "[PHONE]"), (re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+"), "[EMAIL]"), ] llm = ChatOpenAI(model="gpt-4o-mini", temperature=0) def rule_pass(text: str) -> str: for pattern, tag in RULES: text = pattern.sub(tag, text) return text def llm_verify(text: str) -> str: # catches what the rules miss — names, addresses, context-only PII prompt = f"Redact any remaining PII in:\n{text}" return llm.invoke(prompt).content def redact(raw_text: str) -> str: deterministic = rule_pass(raw_text) return llm_verify(deterministic) # same input -> same output
← swipe to see the full pipeline →
ETL Transformation Logic (Python)
Cleaning and refining UK government datasets before loading into PostgreSQL (Project 4).
Core Transformation Function
# Standardize currency and date formats def clean_and_transform(df): # Drop rows where the key identifier is missing df = df.dropna(subset='record_id') # Convert currency to a standard float format df['amount'] = df['amount'].str.replace('£', '') df['amount'] = df['amount'].str.replace(',', '').astype(float) # Standardize all date columns to YYYY-MM-DD for col in ['issue_date', 'expiry_date']: df[col] = pd.to_datetime(df[col], errors='coerce') # Feature engineering: duration in days df['duration_days'] = (df['expiry_date'] - df['issue_date']).dt.days return df
← swipe to see the full pipeline →
FastAPI Endpoint (GCP Cloud Run)
Serving clean data to PowerApps and Power Automate behind the HCLTech automation tool (Project 3).
← swipe to see the full pipeline →
FastAPI Endpoint Code
# main.py for Cloud Run deployment from fastapi import FastAPI, HTTPException from typing import List, Dict app = FastAPI() # BigQuery client connection DATA_SOURCE = get_bigquery_client() # Endpoint to retrieve processed client data @app.get("/api/v1/client_data", response_model=List[Dict]) async def get_client_data(): try: query = "SELECT * FROM client_dataset.latest_snapshot" results = DATA_SOURCE.query(query).to_dataframe() return results.to_dict('records') except Exception as e: raise HTTPException( status_code=500, detail=f"Database error: {e}" )
Web Scraping & Sentiment Scoring
Scraping and scoring sentiment across 200+ sites on an automated schedule (Project 5).
Scrape & Score
# sentiment_pipeline.py — score sentiment across scraped pages from bs4 import BeautifulSoup import requests def fetch_text(url: str) -> str: resp = requests.get(url, timeout=10) soup = BeautifulSoup(resp.content, "html.parser") return " ".join(p.get_text() for p in soup.find_all("p")) def score_sentiment(text: str) -> float: positive, negative = {"good", "strong", "growth"}, {"bad", "weak", "decline"} words = text.lower().split() score = sum(w in positive for w in words) - sum(w in negative for w in words) return score / max(len(words), 1)
← swipe to see the full pipeline →