ICAT Project


ICAT Collaboration Meeting - 25th January 2024



  • Alejandra Gonzalez-Beltran
  • Rolf Krahl
  • Salman Matalgah
  • Louise Davies
  • Gabriel Preuß
  • Viktor Bozhinov
  • Kistry Syder
  • Alex de Maria
  • Marjolaine Bodin
  • Rodrigo Cabezas Quirós
  • Andy Gotz
  • Kevin Phipps
  • Alexander Kemp
  • Malik Almohammad
  • Alan Kyffin

Apologies: ?

Site Updates


Gave presentation/demo of SESAME Data Portal Challenges:

  • Public access to data portal
  • Upgrade OS (currently using CentOS 7)
  • Globus
  • Tape archiving & integrate w/ ICAT/IDS/Data portal
  • Write scientific paper & documentation

Andy: risks with Payara & CentOS? Malik: yes, worried with any risks, ActiveMQ, need to have whole review of system security


Still working to get back to live after incident last June

Andy: could be interesting for Malik Rolf: not sure there's stuff to be learnt - stuff is very chaotic. Not sure how things are going to be going forward. Bit of paranoia right now - a server that's down is safe!

Going to restart service internally first, externally unsure yet

Salman: security assessment of ICAT? Contracting a company in Jordan to do so. Will try & make a report on this Andy: cybersecurity checklist/analysis would be very useful

Andy: how does this work at STFC? Alejandra: basic scanning, concerns around authorisation and exposed data (GDPR risks) Alan: STFC central IT team scans for known vulnerabilities, and external company also scans public facing services. No one auditing ICAT code itself other than developers, other than github advisories Andy: people should share what scans/checks everyone has done - maybe write a paper. Alejandra: ESRF has a report? Andy: yes we've had an audit last year, ICAT not identified as high risk. But our ICAT is not externally accessible. Could ask the company they used about if how they set up ICAT is a risk or not

Malik: concern @ SESAME is that the server contains all the experimental data, so that's why we're cautious


  • Working on converting ispyb to ICAT
  • Implementing sample tracking on Data portal.
  • User can set up experiment parameters & processing parameters
    • Means data analysis pipeline can access these from database
  • Working pushing more processed data from specific domains
  • Moving proposal accounts to user accounts

Working on data2 portal, moving functionality from data1 to 2. It's public but not being advertised yet Example shared with the ISPyB collaboration: https://data2.esrf.fr/investigation/1405067863/datasets

Alejandra: looks good, should talk micro-frontends again in the future to share components. DLS is improving metadata so maybe can support

Andy: request from Hamburg (DESY) to set this up for them. They'd need to setup an ICAT - who to ask about Docker? Rolf, Viktor? Rolf: happy to help, I have only a simple setup, everything on a single host, would like to reconsider in the future

Rolf: DESY? interesting Andy: person we're talking to doesn't feel constrained by SciCat decision - just want something that works Alejandra: there's a central solution? Andy: beamlines can run independently. Not ideal if a beamline needs support, but they do them. Some beamlines have SciCat, some have ICAT Rolf: DESY is "interesting" environment. DESY is large, different groups doing diff things, so no one solution Andy: there's central IT, but some things aren't centralised Alex: P11 (?) beamline e.g. is using ispyb, so natural for them to use ICAT when ESRF does the ispyb -> ICAT stuff

Alex: need ways for new users to ingest data and sync data from user portals


Louise: will remind ISIS to come to this, they now have a bit more effort working on ICAT, they've been doing some work on the ingest system and improving deployment, also applying new search in ISIS dev in time for their next cycle (mid Feb?), equally migrate from Centos7 to Rocky9.


Kirsty: trying to get more metadata into ICAT, short 3-month grad project concetrating on MX beamlines, more information samples, also have been working on funding information.

Andy: what is the state of the DOI service.

Alex: still working on it, made some progress in the UI

Louise: gave a prototype version to DLS, now working on improvements, mainly versioning

Andy: user generated DOIs?

Louise: yes, auto generated will come later

Andy: would be happy to share guidelines for users, how to write good abstracts etc. best practices


Rodrigo: ICAT made public in december ispyb meeting in december, ispyb team @ ALBA taking ESRF approach (ispyb in ICAT)

Andy: since your ICAT is public, should share link in panosc. You made a PR, will follow up Andy: ISIS also isn't there Viktor: yes, aware of the issue, Max emailed, just need to find time to fix Alejandra: Could get funding to finish this work Andy: yes!

Component Updates


Working on next release, 1.3: Metadata ingest workflow. need to work on:

  • documentation
  • XML input format, transformed via XLST to ICAT ingest format. Might need environment info in the XLST transformation e.g. ICAT version, as schema version could affect transformation. WIP, still needs testing

Rolf: I believe Kevin was interested, I'm interested in feedback. best to wait for 1.3 release, but happy to talk about it Kevin: New facility EPAC, in around a years time will be interested. Will need an ingestion pipeline, and this sounded interesting. Rolf: ingest module is marked as experimental currently, but as of next release I consider it ready, and if I get feedback I can remove experimental tag and consider it stable Kevin: not in the near future for us, so don't know when we'll get round to that...


Rolf: Still have pending schema changes that were omitted from v5. I would really like to change sample <-> investigation to many-to-many. Would like to discuss schema changes in the near future Alejandra: can you take ownership/set up some meetings? Rolf: yes, need it outside of collaboration meetings Alejandra: priorities of remaining issues? Rolf: yes, sample <-> investigation issue, but others may have other priorities Alejandra: just send out doodle poll to get the ball rolling

Rolf: sample <-> investigation would be breaking change, so would need all facilities' input to evaluate the impact


Alejandra announced she is leaving her role at STFC Andy: is someone replacing your role? Alejandra: currently? no, responsabilities split across group Andy: need someone to drive the collaboration in Alejandra's stead




€100,000 or €250,000 lump sump @ beginning. Easy process to apply, not sure how hard it's going to be, but aiming for 20% for PaN community. Continue work of PaNOSC & ExPANDs

quite large, if you can link to panosc/expands, you have a stronger case

Trickle down, fund projects and hope it trickles down to all facilities/science

Having someone external to referee is useful, Alejandra can you referee for us?! Alejandra: I think I'll be very busy in my new role! Rolf: difficult, anyone we know we encourage to join in our projects! Andy: yes, can be someone retired, in University, someone really interested in FAIR

First call in March, second call in November. Calls open for 60 days Rolf: I heard 15th March (rumour) Andy: I'm skeptical, but we'll see

Alejandra: if you apply for funding & get it, do you get support? Andy: bit more vague, building competency centres

Andy: Scope is wide: FAIR and related to EOSC & that's about it

Next meeting

29th February



Could have collaboration meetup @ NOBUGS


Rolf: do you have a link to the paper? Andy: we have a DOI: 10.18429/JACoW-ICALEPCS2023-WE3BCO07 but it's not published there yet