Healthcare AI starts with clinical messages you can query. Databricks' Funke parses HL7v2 without FHIR first

Most hospital systems still talk to each other in HL7 version 2: compact, pipe-delimited messages that announce an admission, a transfer, a lab result, a discharge. On the wire, the format is dependable. In an analytics platform, it is miserable: nested segments, repeating fields and local variations that resist ordinary tables. Many healthcare AI projects stall there, long before any model is involved.

Databricks’ answer, published on October 8, is Funke, which it calls “an open-source accelerator that turns raw HL7v2 messages into queryable, Spark-native data”. Written in Python and PySpark, it succeeds Smolder, a Scala library Databricks released in 2021.

What it does

Funke parses HL7v2 messages directly into native Spark types, with no conversion to FHIR in between. Databricks says the parsing is “designed to be lossless” and keeps “the full segment, field, component, and subcomponent hierarchy of the original message”. It is built around Unity Catalog, Declarative Automation Bundles and Spark Declarative Pipelines: deployed as a bundle, ingesting through a declarative pipeline with Auto Loader, and landing raw messages in a bronze table and parsed messages in a silver one.

The demonstration uses a synthetic generator of admission, transfer and discharge events, with test messages from the HL7 v2-to-FHIR project, and turns them into operational metrics. Databricks states that Funke “supports every HL7 message type and version”.

Why skipping FHIR is a trade-off, not a shortcut

FHIR is the modern interoperability standard, and many pipelines convert HL7v2 into it before analysis. That conversion is a mapping, and mappings lose or reinterpret detail. Parsing the original losslessly keeps everything the sending system said, and lets teams decide later what to map.

The cost is that the semantics remain unmapped. A parsed message is a faithful structure, not a clinical meaning: which local code means what, how one hospital’s variation differs from another’s, which field carries the attending physician. Databricks is explicit that Funke “is not an interface engine” and does not replace the clinical and HL7 expertise needed to interpret messages; mapping segments and fields to business concepts stays with the user. FHIR still matters wherever data has to be exchanged with other organizations.

A lossless parse does offer one thing that is easy to overlook: it can be checked. Reconciling parsed tables against raw messages, message by message, is a deterministic test that a lossy conversion cannot pass. In regulated data, that kind of proof from outside the model is worth more than any claim about the parser.

What it leaves to you

Protected health information. The bronze table holds raw messages, which carry names, identifiers and diagnoses. The post does not address how to protect them. Unity Catalog permissions, column masks, retention rules and audit on those tables are the team’s design, and under HIPAA’s minimum necessary standard, most consumers should never touch the bronze layer.

The license. Databricks calls Funke open source, but the repository uses the DB license, which says: “You may not use the Licensed Materials except in connection with your use of the Databricks Services.” That is fine for Databricks customers. It is not the same as a permissive license, and teams that expect to run the parser elsewhere should have legal read it first.

Support. Funke is a Databricks Industry Solutions accelerator. The post does not say it carries product support, so plan to own it like code you adopted.

The order matters

The AI value in hospital operations, such as predicting bed demand, flagging delayed discharges or answering a care coordinator’s question in plain language, sits on top of clean event streams. Admission, transfer and discharge events become census and length of stay through deterministic logic first. Models come after, and they are only as good as the events they read, the same dependency we described in the data path behind agents.

Funke does not make healthcare AI. It removes one of the most common reasons it never starts: clinical messages nobody can query.