No. 31 - Gemini agent launch plus the Lancet AMIE study

No. 31 - Gemini agent launch plus the Lancet AMIE study

On 8 October, Google Cloud launched the Gemini agent at Gemini at Work 2026: a single persistent, cloud-run agent for knowledge work, media and code that spawns sub-agents and can operate as a "coworker" with its own identity and email address. It routes tasks across Gemini models and, per Google, Anthropic's Claude. The emphasis is control: per-agent identity, agent-attributed audit trails, sandboxed execution, an Agent Gateway policy layer and real-time spend caps. Healthcare-specific skills are "coming soon" (Financial Services and Legal are in preview). Highmark Health and Seattle Children's are cited for administrative and clinical support workflows, without metrics. All customer results in the announcement are vendor-reported.

The same day, The Lancet published what its authors describe as the first preregistered study of a patient-facing LLM diagnostic system in real care. AMIE, built on Gemini 2.5 and funded by Alphabet, held pre-visit text conversations with 114 urgent-care patients at one Boston academic practice (98 with complete data) while a physician supervisor watched each chat live. No conversation required a safety stop; supervisors recorded one hallucination and added clinical information in five cases. The adjudicated diagnosis appeared in AMIE's differential within three candidates in 75% of cases and within seven in 90%. Blinded evaluators rated its management plans equal to the primary care physician's in 48% of cases, with physicians rated higher on cost-effectiveness and practicality (both p<0.01).

The evidence is feasibility-grade: single-arm, single-centre, a young and digitally literate cohort, pregnancy and mental-health complaints excluded, a base-model switch mid-study, and a reference diagnosis set by physicians who had seen AMIE's output. The authors state it does not yet support autonomous use. The two announcements sit at different layers: Google has productised the controls to identify, restrict and audit an agent, while the clinical evidence for its patient-facing system covers a prior-generation model under continuous human supervision. The zero-safety-stop result belongs to that supervised workflow as much as to the model. Disclosure: Google funded the study and its authors include Google employees; this issue is produced with Claude assistance, and Anthropic models are among those the Gemini agent can orchestrate.

Implication for the board: governance tooling lowers the cost of deploying an agent but does not establish that it is clinically safe. Before any healthcare-vertical agent touches patient-facing or clinical-support workflows, management should require the supervision model, escalation criteria and evidence on the specific model version in use, and treat "human-on-the-loop" as a design requirement rather than a feature. Watch for the release and content of Google's healthcare skills, the multi-arm randomised trials the AMIE authors call for, and independent, non-Alphabet replication.


Pre-Print Intelligence (arXiv)

Beyond Report Imitation: Clinically Aware Multi-Image Ultrasound Report Generation from Visible Evidence

Brief: CAMEO is a three-stage framework that learns visual-language primitives from ultrasound images, grounds multi-image evidence via cross-view QA, and aligns outputs with clinically aware preference pairs to generate accurate reports. It uses distilled evidence-grounded pairs and preference data to reduce hallucinations and unsupported completions. On the USReport-Distilled benchmark, it improves BLEU-1, ROUGE-1, METEOR and ClinicalScore over prior baselines.

Methodological Integrity: The approach relies on curated datasets (USReport-Distilled, USReport-Pref, PubMedVision-US) which may introduce bias if the source reports are not representative of diverse populations or equipment. Validation is limited to internal benchmarks; external generalization across different ultrasound machines and clinical settings remains untested.

Strategic Implication: If integrated into ultrasound workflows, CAMEO could reduce reporting time and improve consistency, particularly in high-volume point-of-care settings. However, adoption will depend on demonstrating regulatory compliance and seamless interfacing with existing PACS and EHR systems.

Executive Summary: The paper presents a clinically aware multi-image ultrasound report generation method that shows measurable gains in language and clinical metrics. It addresses evidence grounding and preference alignment to mitigate hallucinations.

Innovation: 9/10 | Applicability: 7/10 | Commercial Viability: 8/10


ARISE: Adaptive Agentic Reasoning with Image-grounded Self-Evaluation for Interpretable IBD Assessment

Brief: ARISE introduces a five-stage agentic workflow that guides a vision-language model through hypothesis generation, evidence summarization, refinement, symbolic verification, and final diagnosis for IBD assessment using wireless capsule endoscopy and ultrasound images. The framework improves diagnostic accuracy over baseline VLMs while providing traceable reasoning steps to aid interpretability.

Methodological Integrity: Evaluation relies on two retrospective cohorts with limited sample sizes and no external prospective validation, raising concerns about overfitting and generalizability. The approach still depends on underlying VLM priors, which may harbor biases or hallucinations not fully mitigated by the self-evaluation steps.

Strategic Implication: If validated in larger, multi-center studies, ARISE could reduce specialist workload in gastroenterology and support ambient decision-support in endoscopy suites, but clinical adoption will require regulatory clearance and integration with existing image-management systems.

Executive Summary: The paper presents an agentic reasoning framework that enhances VLM-based IBD interpretation with improved performance and explainability. It demonstrates proof-of-concept on WCE and ultrasound data from two patient cohorts.

Innovation: 8/10 | Applicability: 8/10 | Commercial Viability: 7/10


Knee3DVLM: Dual-Sequence Full-Volume Vision-Language Modeling for Comprehensive Knee MRI Assessment

Brief: Knee3DVLM is a vision-language model that processes full-volume DESS and TSE knee MRI sequences to predict 57 binary MOAKS-derived pathology labels. It uses a dual-sequence architecture to fuse complementary imaging information and was evaluated on subject-disjoint splits from the Osteoarthritis Initiative, achieving ~73% accuracy and ~79% ROC-AUC on a held-out set of 1,074 exams.

Methodological Integrity: The study relies on a single public dataset (OAI) and reports performance on a held-out cohort without external validation or prospective clinical testing. Potential risks include label noise from MOAKS scoring, lack of diversity in patient demographics, and no ablation of sequence-specific contributions beyond the three configurations tested.

Strategic Implication: If validated prospectively, such a model could assist radiologists in structured reporting of knee MRI, reducing interpretation time and variability. However, clinical adoption would require integration with PACS, regulatory clearance, and demonstration of impact on workflow or patient outcomes.

Executive Summary: Knee3DVLM demonstrates that dual-sequence full-volume VLM can achieve moderate accuracy in predicting multiple knee pathology labels from MRI. The work remains at the research stage with limited evidence of real-world utility.

Innovation: 8/10 | Applicability: 7/10 | Commercial Viability: 7/10


Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation

Brief: The paper introduces Missing Modality-Aware Local Temperature Scaling (MMA-LTS), a voxel-wise calibration method that adjusts confidence estimates based on which MRI modalities are missing and local voxel difficulty. Evaluated on BraTS 2020 and FeTS 2024, MMA-LTS improves calibration metrics without degrading segmentation accuracy across various missing-modality patterns. This addresses a key trustworthiness barrier for clinical deployment of multimodal brain tumor segmentation models.

Methodological Integrity: The approach relies on post-hoc temperature scaling requiring a validation set to learn spatially adaptive temperatures and modality-availability tokens; potential risks include overfitting to the validation distribution and limited generalizability to unseen acquisition protocols or tumor types. The evaluation uses standard benchmarks but lacks external multi-institutional testing and ablation of the token learning component.

Strategic Implication: By providing reliable confidence estimates under realistic missing-modality conditions, the method could increase clinician trust and facilitate regulatory acceptance of AI-based segmentation tools in neuro-oncology workflows. However, its impact is confined to the calibration step and does not alter the underlying segmentation model or workflow integration, limiting broader strategic value.

Executive Summary: MMA-LTS is a post-hoc voxel-wise calibration technique that conditions temperature scaling on modality-availability tokens and local difficulty scores to improve confidence estimation in brain tumor segmentation when MRI modalities are missing. Experiments on BraTS 2020 and FeTS 2024 show improved calibration scores while maintaining segmentation accuracy.

Innovation: 7/10 | Applicability: 8/10 | Commercial Viability: 6/10


UltraWorld: Learning Interactive Ultrasound World Models from Untracked Clinical Videos with Acoustic Sampling Map

Brief: UltraWorld introduces a self-distillation framework that learns interactive ultrasound world models from untracked clinical videos by synthesizing action-video pairs via programmable anatomical trajectories and an Acoustic Sampling Map (AsMap). The resulting model predicts future ultrasound observations from local probe actions without needing pose annotations or 3D assets at inference time. Experiments in simulation show improved prediction fidelity and closed-loop planning performance versus visual servoing baselines.

Methodological Integrity: The approach relies on synthetic data generated from a video foundation model and anatomical masks, which may introduce domain shift and limit generalization to real scanner physics and tissue variability. Validation is confined to simulated closed-loop episodes; no real-world clinical trials or prospective user studies are reported, raising concerns about leakage from synthetic training to evaluation and limited evidence of robustness across diverse patient anatomies and equipment.

Strategic Implication: If validated in clinical settings, UltraWorld could reduce the barrier to autonomous ultrasound scanning by eliminating the need for external tracking systems, potentially lowering device cost and expanding point-of-care use. However, commercial adoption will require integration with existing ultrasound platforms, regulatory clearance for autonomous probe motion, and demonstration of tangible workflow benefits such as reduced scan time or increased operator throughput.

Executive Summary: UltraWorld presents a novel self-distillation method for learning ultrasound world models from untracked videos, augmented by an Acoustic Sampling Map to model probe geometry. The work remains primarily simulation-based, with limited real-world validation and unclear path to clinical deployment.

Innovation: 9/10 | Applicability: 6/10 | Commercial Viability: 5/10


templar: agentic induction and evolution of standardized radiology reporting templates from large-scale clinical corpora

Brief: TEMPLAR introduces a multi-agent framework that induces and evolves radiology reporting templates from large clinical corpora using anatomy and diagnostic knowledge graphs. It employs an Induction Agent to derive clinical slots, an Evolution Agent to assemble and refine templates under consistency constraints, and a Clinical Agent for downstream structuring and reasoning. Experiments show improved coverage, fidelity, and diagnostic performance over prior methods.

Methodological Integrity: The evaluation relies on four datasets and compares against baselines like ASTAR and LLMs, but lacks external validation on prospective clinical workflows and detailed error analysis. Potential risks include overfitting to the training corpora and limited generalization to unseen imaging modalities or institutional reporting styles.

Strategic Implication: If successfully integrated, TEMPLAR could reduce manual template authoring effort and improve report standardization across radiology departments, thereby enhancing interoperability and downstream analytics. However, its impact depends on adoption by EHR/PACS vendors and demonstration of tangible workflow efficiency gains in real-world settings.

Executive Summary: TEMPLAR presents an agentic approach to automate radiology reporting template creation and evolution using knowledge graphs and multi-agent collaboration. It reports superior performance on benchmark datasets compared to existing template induction methods.

Innovation: 6/10 | Applicability: 5/10 | Commercial Viability: 6/10


PubMed Gems

UroVision: a new artificial intelligence software for functional magnetic resonance urography evaluation in children with congenital anomalies of the kidney and urinary tract.

Brief: UroVision is an AI-based tool that automates 3D segmentation and functional analysis of magnetic resonance urography in children with congenital anomalies of the kidney and urinary tract, showing strong agreement with the validated ImageJ MRU software for renal volume and moderate agreement for functional parameters, with excellent reproducibility.

Methodological Integrity: The validation used a retrospective cohort of 50 patients (100 kidneys) comparing UroVision to ImageJ MRU via Pearson correlation, Bland-Altman, and ICC analyses; however, the single-center design, lack of external validation, and restriction to a 2D slice for functional assessment introduce bias and limit generalizability.

Strategic Implication: While UroVision can reduce operator dependence and improve workflow efficiency in pediatric urography, its narrow focus on specialized MRU studies limits broader impact unless integrated into wider imaging AI platforms or extended to other modalities.

Executive Summary: UroVision achieved high correlation for renal volume (r=0.99) and moderate correlation for functional parameters (glomerular slope r=0.63, Patlak coefficient r=0.57) versus ImageJ MRU, with excellent inter- and intra-reader reproducibility (ICC 0.87-1.00). Full 3D analysis allowed reclassification of drainage curves in 16 of 23 borderline cases.

Innovation: 5/10 | Applicability: 6/10 | Commercial Viability: 5/10