Selected work

Clinical knowledge with patient-scoped evidence

NYIC leadership experience combining clinical document retrieval, governed analytical queries, and specialized patient-context tools.

Engineering lessons

How can clinical assistance remain anchored to the right patient context?

  • Normalize patient and encounter identifiers before retrieving or querying evidence.
  • Route structured questions, clinical documents, and bounded tools through distinct services.
  • Enforce patient scope at each boundary and retain the basis for professional review.

Context and professional contribution

A clinician’s question can require structured results, narrative notes, and the sequence of events across several encounters. A useful assistant must find the right patient’s information, preserve the timing and source of each observation, and recognize when the available record is incomplete.

This study describes NYIC leadership experience implementing a doctor-facing knowledge copilot using Snowflake Intelligence, Azure Databricks, and supporting Azure agent services. The documented work covered patient-record normalization, semantic modeling, document processing, retrieval, custom tools, orchestration, and access controls. No patient or institutional identity is disclosed.

The implementation supported evidence access and clinical context assembly. It did not transfer professional responsibility for diagnosis, treatment, or care decisions to the model. Its design emphasized source-linked answers and explicit disclosure of incomplete information.

Normalize patient and encounter context

Python and PySpark pipelines in Azure Databricks normalized patient records, reconciled encounter identifiers, and prepared clinical document chunks. Validated tables and document metadata were published to Snowflake on Azure for structured analysis and search. Patient, encounter, and source references were preserved across both platforms.

This identity layer is essential because an otherwise accurate retrieval can become dangerous when attached to the wrong patient or encounter. The system needs to retain the context that makes a result meaningful, including whether it belongs to the current visit, an earlier admission, or a different source record.

Scanned and image-based records were processed with Azure AI Document Intelligence before chunking. Extracted text retained patient identifier, encounter date, document type, page, section, and source metadata. These fields allowed retrieved passages to be traced to the original record.

Route questions to the appropriate evidence service

The doctor-facing application used Snowflake Intelligence. Its Cortex Agent routed structured patient questions to Cortex Analyst, clinical-document questions to Cortex Search, and specialized workflows to custom tools. Supporting Python agents used Microsoft Agent Framework and Microsoft Foundry Agent Service.

The supporting services included managed conversation sessions, persistent memory in Azure Cosmos DB, Entra ID authentication, and Azure API Management policies. These components connected the conversational experience to controlled tool access and a persistent execution context.

Routing instructions specified when to use structured analysis, when document search was required, when to combine tools, and when to ask the doctor for clarification. Response instructions emphasized concise answers, supporting evidence, and disclosure of incomplete information. The workflow therefore had an explicit path for uncertainty.

Make structured questions clinically interpretable

Cortex Analyst operated over secured Snowflake views with row-access policies and dynamic masking. Queries executed within the doctor’s authorized role and patient scope. Unity Catalog permissions governed the upstream Databricks datasets.

Semantic YAML was generated and maintained programmatically. It defined logical tables, joins, measures, dimensions, synonyms, filters, descriptions, and verified queries. Recurring SQL errors and ambiguous clinical terminology were corrected in this semantic model.

This work addressed the gap between a natural-language question and the exact structure needed to answer it. A phrase such as the latest result depends on the relevant test, timestamp, patient, and status. Encoding these definitions makes the query behavior more inspectable and gives engineers a concrete place to correct recurring ambiguity.

Retrieve documents and invoke bounded clinical tools

Cortex Search services indexed the chunked clinical corpus. Chunk sizes, searchable and filterable fields, metadata filters, refresh cadence, and retrieval settings were tuned to return relevant notes, reports, and discharge summaries within the agent’s context budget.

Custom tools supported patient timeline assembly, latest-result lookup, medication reconciliation, care-gap checks, and deterministic clinical and business rules. Typed input and output contracts made these operations predictable for the agent and available for service-level validation.

The application could present tables and charts for laboratory trends, utilization, and medication history. It also honored requests for answer-only responses or suppressed visualizations. This gave the clinician control over presentation while retaining the same evidence and analytical interfaces underneath.

Enforce patient scope across boundaries

Agent and tool role-based access, Snowflake policies, and Unity Catalog permissions were enforced at their respective boundaries. Patient-scope authorization applied to retrieval and custom tools. Search filters were enforced, and direct access to Cortex Search was restricted. Approved supporting agents and MCP endpoints were cataloged in Azure API Center.

These controls addressed a critical architectural issue. Filtering a final response is too late if another patient’s information has already entered the model context. Authorization must accompany the retrieval and tool request that obtains the information in the first place.

The source portfolio also records suggested questions based on recurring clinician needs, including abnormal laboratory results, medication changes, admissions, unresolved care gaps, and changes since the prior visit. These patterns informed refinement of the semantic model and verified-query set.

Evaluation and documented outcome

The documented outcome is an implemented clinical knowledge interface connecting structured analysis, source-linked document retrieval, and specialized patient-context tools. Its intended role is evidence access and professional assistance, with clinical interpretation remaining in the authorized care workflow.

Evaluation for this architecture should examine patient-scope enforcement, temporal consistency, source fidelity, numerical correctness, and handling of incomplete records. Clinicians need to review whether the answer preserves the meaning of the source and supports the intended task.

The transferable lesson is the importance of integrating semantic definitions and access controls with the user experience. A clinical answer becomes useful when the professional can see the supporting record, understand its context, and recognize the limits of what the system was able to establish.