ICAT Project


ICAT Collaboration Meeting 25/05/2023

ICAT Collaboration Meeting 25/05/2023


Alejandra Gonzalez-Beltran, Marjolaine Bodin, Will Taylor, Alex Kemp, Louise Davies, Malik Almohammad, Grabiel Preuẞ, Nicolas Soler, Allan Pinto, Patrick Austin, Alan Kyffin, Kevin Phupps, Matt Richards, Viktor Bozhinov, Rodrigo Cabezas Quirós


Andy Gotz, Rolf Krahl, Alex de Maria

Face to face

Please upload slides to the Indico site (if not already) Will try to upload photo to the website AGB to try to summarise the notes from the meeting (if anyone has ideas, get in touch)

Site Updates

SESAME (Malik):

  • Went to ESRF after F2F, went great, can now see all the metadata
  • Have ICAT server running with all metadata
  • Not enough time to finish ingestion for all data
  • After returning, wrote python code for collecting all data and putting in the portal
  • Now have it running, but have issue: still using Topcat, need support for newer UI
    • Either DataGateway or Datahub would be valid choices
    • Rolf also facing this decision
    • Discussed with HZB
    • Compatability issues with Red Hat 8
  • Still using 4.8, need to update to version 5

ESRF (Marjolaine):

  • ISpyB collaboration meeting, ingesting MX and CryoET data into ICAT + dedicated UI
  • Will start to work on ingesting paleontology data, existing portal is old
    • Similar to approach to the human organ atlas
  • Certification (CTS) Core Trust Seal approved

HZB (Gabriel)

  • Rolf on leave, and have had leave recently so not much to report
    • IDS work to come
    • More detailed schedule to come

ISIS (Will)

  • No updates

DLS (Kirsty)

  • No updates
  • STFC: still working on open data/DOI minting work

ALBA (Rodrigo)

  • Started working on ingestor
  • Had maintenance on K8s cluster
  • Preparing another beamline to ingest ARPES data in ICAT
  • Trying to use metadata standards, also talking to bealine scientists
  • SO far 2 beamlines ingesting ("incompletely") to ICAT
  • Want to use ARPES as the case study for further beamlines

Sirius (Allan)

  • Running the tutorial for the ICAT
  • Try to follow it but found some problems
    • Alejandra: please report issues with current documentation via mailing list or Slack
      • If documentation is on GH, feel free to contribute directly
    • Malik: working on issues encountered for RH7/8 distros, from installation to authentication
  • Found ESRF repo icat compose, was able to use that start a demo which is working fine, version 4.9??
    • Should this be moved to GitHub, with the other "core" components (at least a link in the icat web site)
  • Next step is to ... currently migrating VMs to Docker containers, saw that many repos are using ... are we using docker composers or is the aim to have it running in production?
    • Rodrigo: running everything in their K8s cluster
    • Documentation recommends setting up a VM - can there be a containerised version - think this would be easier for people
    • ALBA has a TODO list item to share their docker/k8s config with the ICAT collab - just want to make some changes first
    • Rolf has started to work on an ICAT mini docker - don't think it's finished yet, expect a mailing list announcement

Component Updates

ICAT Manager (Will Taylor)


  • Patch version released for both ICAT 4 (4.11.2) & 5 (5.0.1)
  • Resolves an issue re: authorizing one-to-many/many-to-many relationships
    • This is considered a vulnerability
  • Also switches scripts to use Python 3 in 5.0.1 only

Payara 6

  • ICAT 5 only runs in Payara 5, which is out of support
  • A lot of breaking changes due to supported libraries, but no functional change
  • No change in schema between 5 and 6
  • Haven't completed testing in anger
  • Affects multiple components as well

Standalone logbook

  • Marcus at HZB working on it
  • Have a running prototype
  • Based on ESRF's work that's currently integrated into Datahub


South Africa Conference

  • Andy submitted abstract for Cape Town conference
  • If accepted, will become a paper to describe the status of ICAT
  • Every facility can contribute with how they are using it

NFDI meeting (DAPHNE?)

  • Discussed some of their work (including at HZB) for better management
  • Some facilities using SciCat
  • Focus on standardizing between facilities
  • Discussions on logbooks also relevant


  • Q: there are many out there, what would be considered "official"?
    • A: the icat manual repo, markdown based tutorials
    • Might not be up to date, but is closest we have
    • Stuarts gitbook: definitely out of date, will still be on Payara4
    • SHould it br easier to find this manual? Link/copy directly to the website
    • Malik: we already use every service from https://repo.icatproject.org/, a lot of documentation - find this very useful (should this also be more prominent)


  • Out of date - reference job portal, topcat and other outdated components
  • In the icatproject.github.io repo is the source
  • Open pull requests for improvements, contributions welcome
  • Issues asking for new content
  • In parallel to content changes, should we update the look
  • Kenneth, Louise currently assigned to review
  • Include notes from collab meetings in future