ICAT Project


ICAT Collaboration Meeting 27th June 2024



  • Andy Gotz
  • Louise Davies
  • Kirsty Syder
  • Kevin Phipps
  • Malik Almohammad
  • Santhosh Anandarama
  • Viktor Bozhinov
  • Alex de Maria
  • Alan Kyffin



Kevin: is anyone else attending? Alex? Alex: was interested, but didn't hear anything since initial email

Andy: should be able to attend remotely? Louise: yes, there's a remote attendance option on the indico

Andy: Parameters: Does SciCAT have nested? Louise: yes, think so. However, they mentioned having trouble with theirs - hard to assert/search parameters when they're so flexible.


Andy: is this fixed STFC: yes, apologies this was broken by migrating VMs to upgrade the OS. External network configuration was broken but it worked internally.


Need to create an indico page to get numbers of attendance so room can be organised.

STFC will send a couple of people.

Andy: Register ASAP: only 160 seats & already 120 potentially filled by speakers Kevin: when does registration open? Andy: open now https://indico.esrf.fr/event/114/registrations/

Can just register just for satellite(s) or the main conference as well.

Multiple talks from ICAT people at main conference (5) : https://indico.esrf.fr/event/114/search?q=icat

Site Updates


No big updates.

Finished cybersecurity assessment. After, addressed weakness points - especially glassfish. Will share report at next meeting most likely.

Still only accessible on internal network

Plan to start using DataPortal with the public next week.

Andy: how many other sites using glassfish? Rest mostly using Payara Alan: shouldn't really make a difference

Main problem is all services running in same environment. Separate DataPortal onto separate VM is likely safer.

WAF (Web applicaiton firewall) - ESRF looking into implementing


Processed data - 10 beamlines using. High demand for this, initially macromolecular crystollography, but expanding to more techniques. Happy with performance - going smoothly and faster than ispyb.

Collaborate with DESY for crystallography. Going to go over and help them set it all up including ICAT. Thinking about packaging to make this easier.

Andy: should get extra money for that, maybe EOSC?

Andy: feedback from scientists re: electronic logbooks. Difficult as they compare with google docs! (high bar)

Malik: Would be interested in Alex' collaboration - they have tomography beamlines.


Santhosh: user office integration is complete - in testing. Andy: is this a new user office software? Santhosh: no, migration from SOAP to REST Andy: thought new user office - thought they were interested in using ESS UO. Louise & Kevin: Proposals system has been migrated and is using ESS solution.

Andy: chatbots integrated with Data portals - Elettra already have good feedback on their chatbot. Alex: user wants to browse data via chatting. LLM solution (Mistral) is quite promising. Integration with logbook - can ask what happened e.g. overnight One issue is authorization


Had demo of the DOI functionality from Louise & Patrick. Slowly working through approval from executive and legal. Has some agreement on only raw data

Also, Rocky 9 upgrades.

Andy: NOBUGS2024 poster on legal issues from DLS Kirsty: yes, someone using LLMs/ChatGPT to analyse abstracts. Interested to see the poster...

Component Updates


Reverted a change which caused incompatability between new ICAT snapshot and python-icat


download statistics

DAPHNE: DOI landing page download statistics Louise: could do basics with google analytics - we have basic page tracking Alan: we use 3rd party service for some of our other services

Kevin: we have an admin download page in DG to track what the current retrieval status

Kevin: we do have an apprentice working on an ICAT dashboard project We did have old dashboard, but stopped working on it - were some GDPR concerns

DOI stuff

Andy: Data Portal shows users emails e.g. on summary page Louise: we don't - don't want to get into that can of worms! Andy: meeting at ESRF to discuss it

Andy: datacite API - old one (schema 3) depcrecated, must move to schema 4 before end of the year Louise/Kevin: new API DLS mentioned is using new one, some legacy services using old one


Alex: difference between data volume between DB filesize and filesize on disk. If different then download could fail. We now check both DB filesize and disk size. We've fixed it.

Andy: tar vs zip - users had requested to use tar. Kevin: Should be able to use tar - as either config option or api param perhaps Also, zip compression option doesn't work - DLS user pointed this out for us