Skip to content
This is NOT an official site of the City of New York. Click here for official AI reports.

NYC Office of Technology and Innovation

NYC - DTPR AI Register

Back to your search

AI-Assisted Foodborne Illness Detection from Social Media

What it is for

Risk Assessment & Triage
  • The system scores individual Yelp restaurant reviews with a "sick score" to assess whether each review likely describes foodborne illness, enabling epidemiologists to triage and prioritize follow-up on potential outbreak-linked restaurants.
Healthcare
  • Supports the public health function of identifying and investigating restaurant-associated foodborne disease outbreaks in New York City to protect population health.
Run by
Department of Health and Mental Hygiene (DOHMH)
Where
No fixed location
Kept
Retained Annotations and training data are retained and periodically updated to improve classifier performance; specific retention period not publicly disclosed
Shared with
Accountable organization, Vendor

What it is for

Risk Assessment & TriageHealthcare

This system uses a text classifier to scan Yelp restaurant reviews and assign a "sick score" indicating the likelihood that a review describes a foodborne illness. DOHMH staff then review high-scoring entries and, where appropriate, reach out to reviewers and combine findings with 311 complaint data to detect restaurant-associated disease outbreaks more efficiently. As of January 2025, the system has been unable to receive data due to an unresolved technical issue. Members of the public who post Yelp reviews of New York City restaurants may have their review analyzed and may be contacted by DOHMH.

What it collects and what happens to it

Data taken in

About behaviour
Identifiable data
  • Yelp restaurant reviews posted publicly by individual users, pulled from Yelp's privately available API. Reviews contain user-authored text describing dining experiences including potential illness symptoms. User identities are linkable via Yelp profile for follow-up contact.

Processing

Classification & Prediction
  • A text classifier assigns a numeric "sick score" to each Yelp review based on the presence of foodborne illness keywords (e.g. "vomit"), indication that multiple people became sick, and whether a described incubation period is consistent with foodborne illness. Reviews meeting or exceeding a threshold score are flagged for human review.

What it does

Deciding (Analytical AI)
Human decides
  • The classifier scores each Yelp review and flags those meeting a sick-score threshold; DOHMH foodborne disease epidemiology and environmental health staff then review and annotate each flagged entry to determine whether it represents a genuine foodborne illness complaint before any action is taken.

Outputs

A recommendation or prediction
Anonymized data
  • Each processed Yelp review receives a "sick score" indicating the probability it describes a foodborne illness. Reviews with a score at or above the threshold are surfaced to DOHMH staff as candidates for follow-up investigation. The score is advisory; human staff make the final determination.
Sensitive personal information
Identifiable data
  • When a flagged review is confirmed by staff as a foodborne illness complaint, a Yelp message is sent to the specific reviewer requesting they contact DOHMH. The reviewer's Yelp identity is used to make this outreach, rendering the output identifiable.

Run by

Department of Health and Mental Hygiene (DOHMH)
  • DOHMH's Bureau of Communicable Disease, Office of Environmental Investigations, and Division of Informatics and IT & Telecommunications deploy and operate the classifier, review its outputs, and make decisions about contacting individuals who may have experienced foodborne illness.

NYC AI Register — DOHMH Foodborne Disease Outbreak Detection

Built by

Columbia University Department of Computer Science
  • Columbia University professors and doctoral students developed and maintain the Yelp text classifier in a no-cost engagement. The collaboration was previously supported by grants from the Alfred P. Sloan Foundation and the National Science Foundation.

NYC AI Register — DOHMH Foodborne Disease Outbreak Detection

Kept for

Retained Annotations and training data are retained and periodically updated to improve classifier performance; specific retention period not publicly disclosed
  • Training data consisting of annotated Yelp restaurant reviews is retained and periodically updated with new DOHMH staff annotations. The register does not specify a fixed retention period for review data or sick scores.
  • Duration: Annotations and training data are retained and periodically updated to improve classifier performance; specific retention period not publicly disclosed

Shared with

Available to the accountable organization
  • Sick scores, flagged reviews, and staff annotations are accessible to DOHMH's Bureau of Communicable Disease, Office of Environmental Investigations, and Division of Informatics and IT & Telecommunications for outbreak detection and classifier improvement purposes.
Available to vendor
  • Columbia University Computer Science staff who maintain the classifier have access to the training data and classifier outputs as part of the collaborative no-cost engagement for model development and maintenance.

Stored

Stored locally
  • The register implies data is processed and stored within the DOHMH and Columbia University operational environment in New York City, though specific storage infrastructure is not described.
  • Duration: Not publicly specified
How to read the colours

Can it identify you?

Anonymized data
Data about people with the link to who is broken. Stripped of identifiers, blurred, aggregated, or noised so this system can’t reasonably tie a record back to an individual.
Pseudonymous data
Each person’s data is tied to a token (hash, ID, template) that lets this system recognise the same person across events, but the token itself doesn’t reveal a name. Reidentification is possible with extra information.
Identifiable data
The data either contains a direct identifier (name, address, account name, recognisable face or voice, plate number) or carries a token this system uses to look up legal identity during processing.

Who completes the loop?

Human decides
This mode suggests; a person decides what to do next. The AI is always advisory — a human is in the loop on every decision. Example: a triage tool ranks cases for a clinician who chooses which to see first.
Human executes
This mode decides; a person carries out the result. Example: an optimizer plans the day’s trash-collection routes, and drivers run them.
Autonomous
This mode decides and acts on its own. No person reviews each decision or carries out the resulting action.

Definitions from the DTPR standard. Amber is about your data, violet about who decides. The fuller the shape and the deeper the colour, the more identifying the data or the less a person is involved.

What you can do

Ask about this system

Questions go to the NYC Office of Technology and Innovation, not the vendor.

Your rights

  • Right to Be Informed of AI UseMembers of the public who post Yelp reviews of New York City restaurants should be aware that DOHMH may analyze their reviews using an automated text classifier to detect potential foodborne illness complaints. If your review is flagged, you may be contacted via Yelp by DOHMH requesting that you report your experience directly. This AI use case is disclosed in the NYC Automated Decision Systems register.
  • Right to a Human ReviewNo automated action is taken solely on the basis of the classifier's output. DOHMH foodborne disease epidemiology and environmental health staff review and annotate every flagged review before any outreach is initiated. Human staff make all final decisions about whether to contact a reviewer.

Risks and safeguards

  • Civil liberties harmThe system monitors publicly posted Yelp reviews without the explicit consent of reviewers for surveillance purposes, raising concerns about passive data collection from individuals who did not intend to interact with a public health authority.Safeguard: outputs are reviewed and annotated by DOHMH staff before any contact is made; the system is used only for public health outbreak detection; data are not shared with law enforcement; and follow-up contact is limited to a voluntary request to contact DOHMH.
  • Reputational harmRestaurants flagged by the classifier as potentially associated with foodborne illness could face unwarranted scrutiny if scores are inaccurate.Safeguard: all classifier outputs are reviewed and annotated by DOHMH epidemiology and environmental health staff before any investigative or enforcement action, and training data is periodically updated with staff annotations to improve accuracy.