Context

In this assignment, you will synthesize the core NLP workflows learned throughout this course. You are tasked with designing and writing a Python script or Jupyter Notebook (.ipynb) that models an end-to-end NLP pipeline. This pipeline must process long unstructured text, index it for document search, retrieve relevant segments to answer natural language questions, extract key organizational entities, and analyze their associated sentiment.

Scenario & Objective

Your task is to build a prototype system in Python that simulates processing a batch of long-text documents (such as financial statements, news feeds, or Reddit data). The prototype must incorporate:

Text Preprocessing & Unicode Normalization: Normalizing raw inputs using standard Unicode forms (e.g., NFC or NFKC).
Document Indexing & Haystack Pipeline: Setting up an Elasticsearch document store, a Sparse (BM25) or Dense (DPR) Retriever, and a Transformer-based Reader.
Long-Text Segmentation: Structuring a window-based classification approach to handle the BERT 512-token model constraint.
Named Entity Recognition (NER) & Filtering: Extracting 'ORG' entities using spaCy or roBERTa, and applying an entity blacklist.
Sentiment Integration: Analyzing the extracted context around the entities using a prebuilt Flair or Hugging Face model.
Evaluation Metrics: Assessing the QA model output using Exact Match (EM) and ROUGE metrics.

Deliverables

Please submit a single Python file (nlp_pipeline.py) or a Jupyter Notebook (nlp_pipeline.ipynb) containing:

A documented Python structure using the relevant frameworks (Hugging Face Transformers, Haystack, spaCy, and/or Flair). Since you may not have active local servers running during submission, you can write structurally complete code blocks with comments indicating the execution flow, initialization parameters, and expected tensor/data shapes.
Markdown notes or code comments explaining your architectural choices, including the specific retriever-reader configuration (e.g., BM25 vs. DPR) and the formulas/logic behind ROUGE and Exact Match evaluation.

Submission Checklist

[ ] Preprocessing step uses Unicode normalization (NFC/NFKC).
[ ] Code drafts a functioning Haystack Retriever-Reader pipeline.
[ ] A windowing logic is outlined to prevent exceeding the BERT 512-token limit.
[ ] Entities ('ORG') are extracted and run through a frequency and blacklist filter.
[ ] ROUGE and Exact Match evaluation metrics are configured.