10:03:41 From Goran Nenadic to Everyone : Programme: http://healtex.org/healtac-2021/programme/ 10:04:09 From Goran Nenadic to Everyone : Please feel free to interact using the chat, zoom reaction buttons, and on twitter: 10:22:37 From Mat Rawsthorne (he/him) to Everyone : apologies if this is a stupid question but is there is an outpatient clinic equivalent of ClinicalBERT which might cover notes/letters relating to less intensive, long term conditions? 10:25:19 From Mat Rawsthorne (he/him) to Everyone : could this approach be used (with permission) for public health surveillance to identify emerging phenomena? 10:27:10 From Paul Rayson to Everyone : Thanks for the interesting talk Samantha! Is there much noise in the data from searching for *itis? e.g. it tends to get used quite widely for other non-medical terms e.g. end-of-term-itis? 10:31:46 From Luke Slater to Everyone : don't think clinbert used negation. i think usually they say that they will capture negation context in the embeddings somehow 10:32:26 From Luke Slater to Everyone : could be an interesting experiment to train two sets of embeddings... one for negated one for affirmed, compare 10:32:41 From Goran Nenadic to Everyone : And yet, around half of all mentions of clinical entities in letters/notes are negated. 10:32:46 From Mat Rawsthorne (he/him) to Everyone : Samantha what has been the impact of the work on the clinicians e.g. have they changed the way they ask about symptoms? 10:35:28 From Samantha Pendleton to Everyone : @Paul, we used "itis" as a lot of inflammatory conditions have this suffix - and the forum we extracted from had topics, e.g. "general health" or "blepharitis" and so we only used the individual posts which were posted in the specific topics of interest - so this way we could avoid noisy data 10:36:56 From Samantha Pendleton to Everyone : @Mat interestingly in the OCIMIDO project, when the domain expert looked at the potential synonyms they were surprised at some of the patient-preferred terms used 10:37:10 From Paul Rayson to Everyone : @Samantha - thanks, sounds like a good approach 10:37:43 From Mat Rawsthorne (he/him) to Everyone : thanks Samantha - I hope you can capture that impact and publish it! 10:38:05 From Samantha Pendleton to Everyone : Thank you all! :-) 10:45:13 From Luke Slater to Everyone : Hi Matus, nice talk, thank you. are you familiar with semantic similarity measures? it sounds very similar to what you're trying to achieve with set-based hierarchical evaluation, and there's a lot of research in this area. do you know how they differ? you might also be interested in random walks for representation of ontology axioms/structure in vectors/embeddings. 10:53:09 From Hang Dong to Everyone : Hi Matus, I wonder can CoPHe be applied to evaluate manual coding? I would assume that human coders are less likely to pick multiple codes under a same parent for the same patient. (maybe set-based and CoPHe are not that different in this case). Just my random thought. 10:55:04 From Matus Falis to Everyone : HI Luke, thank you for the question and advice. I do not believe I am familiar with the measure you describe, but it does sound interesting! 10:55:51 From L.SLATER.1@BHAM.AC.UK to Everyone : well, it's more a set of measures, that uses structural and information content measures, to measure similarity between two concepts in any ontology. maybe take a look into https://www.semantic-measures-library.org/sml/ 10:56:28 From Matus Falis to Everyone : Hi Hang. Sure, as long as a vector of leaf codes is provided, CoPHE can evaluate it, regardless of whether it is a prediction from a model or if it is coded manually. 10:56:56 From L.SLATER.1@BHAM.AC.UK to Everyone : https://academic.oup.com/bib/advance-article/doi/10.1093/bib/bbaa199/5922325 might be a good rea 10:56:57 From L.SLATER.1@BHAM.AC.UK to Everyone : d 10:57:22 From L.SLATER.1@BHAM.AC.UK to Everyone : well, it is a good read, but i mean might be suitable for you.. 10:57:43 From Patrick Schrempf to Everyone : Thanks for the presentation Matúš! 10:58:35 From Goran Nenadic to Everyone : Can speakers please move to the Speaker’s room if they want to check their slides please? 10:58:56 From Matus Falis to Everyone : Thank you Luke. What I am assuming is that where I was only using structural information - a thing whose correctness I can attribute to medical professionals, the semantic similarity would also be using the semantic rep of the label text. Interesting and adventurous! 11:01:14 From PJ to Everyone : Nice if you can go surfing during the beaks ! 11:01:28 From PJ to Everyone : Breaks 11:01:40 From L.SLATER.1@BHAM.AC.UK to Everyone : not quite - so originally semantic similarity was created for comparing words and labels, but in the context of ontology it's usually referring to using measures of structural similarity (and also information content, e.g. term specificity or probability of appearance in a corpus) - so i think the same 'class' of measures that you are investigating! 11:02:01 From L.SLATER.1@BHAM.AC.UK to Everyone : involving semantic similarity of label text would be a really interesting progression, though. also for annotation in general 11:11:49 From Mat Rawsthorne (he/him) to Everyone : Hi Aurelie what has been the response of clinicians to this work e.g. has it changed their assessments? 11:13:50 From Mat Rawsthorne (he/him) to Everyone : my experience of anti-psychotics is that they can cause cognitive impairments ('brain fog') - were you able to control for that? 11:13:54 From Owen Pickrell to Everyone : Would there be a possible bias in that patients with more hospital admissions/problems are more likely to get cognitive symptoms recorded as opposed to those who remain out of hospital? 11:15:28 From PJ to Everyone : Am I right in saying you had just 2000 expert-annotated notes per syndrome to fine-tune the BERT model ? 11:16:15 From Kristof Anetta to Everyone : @Aurelie - how scalable is this approach? How difficult could semi-automatic annotation be? 11:16:16 From Mat Rawsthorne (he/him) to Everyone : Hi Owen thanks for raising that - why I was asking about an outpatient version of clinicalBERT in the first session! 11:16:46 From PJ to Everyone : COol ! 11:16:55 From Mat Rawsthorne (he/him) to Everyone : really cool work thanks Aurelie! 11:17:27 From Jenni Ajderian to Everyone : really interesting thanks Aurelie! 11:19:58 From Kristof Anetta to Everyone : Thank you! 11:20:16 From Mat Rawsthorne (he/him) to Everyone : Aurelie will you be sharing your slides? 11:20:34 From Aurelie Mascio to Everyone : There you go :) https://docs.google.com/presentation/d/1xQEewTUnOT21L8a7SqmDMpeoPswCkeDens5OanPKSdo/edit?usp=sharing 11:23:11 From Mat Rawsthorne (he/him) to Everyone : if there is alexithymia would that lead to more misunderstandings and conversation ruptures? 11:30:03 From Mat Rawsthorne (he/him) to Everyone : what was the response of people with GAD to your work? 11:31:55 From Jenni Ajderian to Everyone : was the expressive writing exercise always a diary entry? or did they answer a specific q? 11:32:12 From rturrisi to Everyone : Thank you for the talk! I’d like to ask if you think that it may be possible also to distinguish between depression and GAD from text 11:34:13 From Mat Rawsthorne (he/him) to Everyone : there was a UK project led by Mark Brown called "A day in the life" which involved a large number of diary entries by people living with mental health issues. Not sure whether you could extract demographic data from that to also test the generalisability of your work 11:34:49 From Mat Rawsthorne (he/him) to Everyone : https://www.centreformentalhealth.org.uk/blogs/day-life-mark-brown 11:35:42 From Mat Rawsthorne (he/him) to Everyone : I have a scrape of that data (and my own analysis) if you are interested 11:36:34 From Laurens Rook to Everyone : Dear all, thanks for the questions and the valuable links (which we will definitely check out)! 11:36:59 From Mat Rawsthorne (he/him) to Everyone : very interesting talk - thank you Laurens! Are you able to share your slides? 11:40:36 From Laurens Rook to Everyone : As for the question on Depression vs. GAD: The developers of the GAD-7 also produced a Depression scale (PHQ-8) with similar psychometric properties. We will use these two scales to assess self-reported differences. As for unique text entries: In this study, text entries on GAD were only significant for people low on BAS. We hope to find for Depression significant correlations between words and high BAS in future runs… Hope that answered your question? 11:52:17 From Dage Särg to Everyone : why should we use "less advanced technologies" for lower-resource languages? bert etc can be trained when you have a dataset and that you have? 11:53:48 From Gregory Kell to Everyone : Have you investigated the use of transfer learning i.e. English->Polish? Or maybe even from more similar languages, i.e. Russian->Polish? 11:56:04 From Micheal to Everyone : yes, cross-lingual modelling would be helpful 11:56:18 From Paul Rayson to Everyone : Thanks everyone! 11:58:07 From Elizabeth Ford to Everyone : Hi Natalie Fitzpatrick! Would you like to share the first slide for the panel session now? 11:59:57 From Luke B to Everyone : Thanks for the talk Kristof! Would you be willing to share your slides? I don't know if this will help you or not but https://ranzato.github.io/ actually suggests bigger models trained on more data is helpful for low-resource language modelling. Give him a look if you have time. 12:00:02 From Kristof Anetta to Everyone : @Dage - we will definitely work with BERT, the biggest problem is having sufficient vocabularies in the first place. I guess the "less advanced" part would be manual vocabulary creation. 12:03:13 From Kristof Anetta to Everyone : @Gregory - that is definitely an interesting avenue of research, we were considering some translation-based transfers from knowledge in English, if you have anything else in mind I am all ears! 12:05:05 From rturrisi to Everyone : @laurens yes, thank you.. very interesting! 12:05:34 From laurensrook to Everyone : thank you! 12:08:35 From Elizabeth Ford to Everyone : Web links to papers mentioned: 12:08:39 From Elizabeth Ford to Everyone : Clinical benefits: https://www.frontiersin.org/articles/10.3389/fdgth.2021.606599/full Patient views on data sharing and possible harms: https://wellcomeopenresearch.org/articles/3-6/v2 Healtex citizen’s jury https://jme.bmj.com/content/46/6/367 TexGov paper https://www.jmir.org/2020/6/e16760/ 12:11:29 From Kristof Anetta to Everyone : Sorry for the overflow from the preceding session. @Luke - thank you for the link, I will check it out. My slides are here: https://we.tl/t-JzFk0uylpN 12:12:48 From Katrina Davis to Everyone : I love that Debbie "it needs to be above the waterline, not just in academic journals" 12:17:14 From Luke B to Everyone : @Kristof Thank you! 12:26:53 From Katrina Davis to Everyone : Alongside engaging patients, we also need to engage clinicians so that they are able to have informed conversations with patients about what happens (or could happen) to their data. Do people know any good models for this? 12:44:46 From Natalie Fitzpatrick to Everyone : I think that is such a great idea Rebecca 12:45:48 From Colin Wilkinson to Everyone : How do we ensure the free text in people's electronic health records is accurate, thus forming a good basis for research, when people can't even see their records, without creating a huge bureaucracy, and allowing patients the option to edit directly (which would be unwise - records have to be agreed between patient and clinician)? (Having seen mine, they were littered with errors, some serious) 12:53:11 From Katrina Davis to Everyone : https://en.wikipedia.org/wiki/Amir_Hannan 12:55:29 From Amanda Roberts to Everyone : Free text is often the only way researchers can dig into "minor condition" details where coding is not able to deal with the needs Should free text be such an apparent jumble of mixture of straight data and privileged conversations etc 12:56:24 From Goran Nenadic to Everyone : Access to patient's own data should be a no brainer. But coming back to access to large-scale datasets for NLP training - what do the panel see as the key risks? Is it NLP researchers who have access? Is it annotators who produce tagged data? Is it the insecure environment? 12:58:23 From PJ to Everyone : How do you get a jury that includes turnip growers ? 12:58:45 From Alex Handy to Everyone : @Goran - an additional comment on the risk, for me as a researcher its the potential insecurity of the environments and working processes. My concern is that I could accidentally expose sensitive data even if taking many precautions not too. 12:58:48 From Phil Booth to Everyone : Quick answer to Luke: Differential Privacy trades off privacy vs accuracy (according to how you set the epsilon value) - so it really isn't as useful for health data research that *needs* to be on individual-level data. Tech like homomorphic encryption, being explored around processing genomic (DNA) data is more appropriate 12:58:49 From Katrina Davis to Everyone : To Goran: It is the GP who tells the patient that his data will not be used for research, when actually it is. Confidence is quickly lost! 12:58:50 From Gianpiero CELINO to Everyone : Hi @goran, my response would be all of these! We need make each element as safe as is possible and even then if that is not enough then we need to be prepared to say no. 13:00:11 From Luke B to Everyone : @Phil Thank you very much! 13:00:42 From Phil Booth to Everyone : To Goran - many of these issues *could* be addressed in a TRE-only world. 13:01:24 From Goran Nenadic to Everyone : Thanks all, all good and valid points - happy to continue with the chat during lunch! 13:01:29 From Debbie Keatley to Everyone : agreed Phil 13:05:15 From Natalie Fitzpatrick to Everyone : Liz please talk! 13:05:23 From Natalie Fitzpatrick to Everyone : Can't you hear me? 13:07:48 From Natalie Fitzpatrick to Everyone : Following Liz's update on our plans on governance and consented free text data bank, we'd love to hear from you - please feel free to contact either Liz (E.M.Ford@bsms.ac.uk) or myself n.fitzpatrick@ucl.ac.uk 13:09:17 From Elizabeth Ford to Everyone : It sounds so interesting Anoop! 13:11:39 From Anoop Shah to Everyone : Job opportunities for software developers to work on NIHR AI funded UCLH/UCL/GOSH project to develop NLP at the point of care: http://bit.ly/gr8_senior_software_developer, 13:11:43 From Noel Kennedy to Everyone : LIES! It’s just overcast in london :) 13:12:07 From Goran Nenadic to Everyone : :) It will rain during the lunch break! 13:14:06 From André Bittar to Everyone : Photo competition - upload your photos here!
https://drive.google.com/drive/folders/1TmDk59yjSB-xD-9_m2PVsbUmcht1KHI7?usp=sharing 13:14:32 From Angus Roberts to Everyone : From 12:00 - 18:00 each day, you can use gather.town for networking during breaks and after the conference: 13:14:34 From Natalie Fitzpatrick to Everyone : No one has ever wanted to pay me for watching me eat my lunch before :) 13:15:31 From Ayath Ullah to Everyone : Can someone send a copy of this chat to us after the conference today 13:28:40 From Natalie Fitzpatrick to Everyone : Hi @ANdre it says we don't have permission to edit the photo googledoc (and upload photo) 13:33:54 From André Bittar to Everyone : Oh! I’ll try to change that thanks for letting me know @Natalie 13:39:52 From André Bittar to Everyone : HI all, updated link to upload your photos! https://drive.google.com/drive/folders/1TmDk59yjSB-xD-9_m2PVsbUmcht1KHI7?usp=sharing 13:46:22 From David Chandran to Everyone : Reminder of gather.town link: https://gather.town/i/kJfi6h0e 13:55:43 From David Chandran to Everyone : If anyone wishes to test their slides please go to the speakers room 14:02:00 From André Bittar to Everyone : Link to upload photos: https://drive.google.com/drive/folders/1TmDk59yjSB-xD-9_m2PVsbUmcht1KHI7?usp=sharing 14:26:12 From Mat Rawsthorne (he/him) to Everyone : cao 2019 https://arxiv.org/abs/1910.12038 Sawhney 2020 https://www.aclweb.org/anthology/2020.emnlp-main.676.pdf Sawhney 2021https://www.aclweb.org/anthology/2021.eacl-main.205/ 14:28:40 From Mat Rawsthorne (he/him) to Everyone : sorry this is the correct 2020 Sawhney one: https://www.aclweb.org/anthology/2020.emnlp-main.619.pdf 14:30:30 From Mat Rawsthorne (he/him) to Everyone : https://link.springer.com/chapter/10.1007/978-3-030-10997-4_25 14:38:15 From laurensrook to Everyone : Very interesting research! Thank you! 14:40:52 From Aicha Chorana to Everyone : That was interesting. Thank you! 14:41:44 From Helena Ariño Rodríguez to Everyone : Is the approach for change detection in longitudinal sequential tasks the same for single events (such as suicidality) or recurrent events (disease relapse) at a patient level? 14:41:52 From laurensrook to Everyone : When we are trying to identify key moments of change (for PANAS), we end up doing qualitative coding (for which we need multiple coders, etc.). Is this something you recognise? 14:42:13 From Aicha Chorana to Everyone : Are the slides accessible? 14:43:58 From rturrisi to Everyone : How do you think this type of research may be applied in real life? Let’s consider the case in which you diagnose a mental state from twits, can you “notify” the subject or aren’t you allow to give him feedback for privacy reasons? 14:44:15 From Helena Ariño Rodríguez to Everyone : Yes, thanks 14:44:34 From Ghada Alfattni to Everyone : interesting work, thanks. How are you going to evaluate the models? 14:45:17 From laurensrook to Everyone : Ah…. so there is no escape to this ;-) Thank you! 14:45:22 From Mat Rawsthorne (he/him) to Everyone : are any of your coders people with lived experience of the conditions you are studying? 14:50:45 From Mat Rawsthorne (he/him) to Everyone : thanks Maria, fascinating! 14:50:58 From laurensrook to Everyone : what was the link for the photo upload again? 14:51:08 From André Bittar to Everyone : Link to upload photos: https://drive.google.com/drive/folders/1TmDk59yjSB-xD-9_m2PVsbUmcht1KHI7?usp=sharing 14:51:15 From laurensrook to Everyone : THX 14:51:21 From Maria Liakata to Everyone : I am really glad you enjoyed it ! 14:53:12 From Goran Nenadic to Everyone : And if you want to chat and socialise in the gather.town: https://gather.town/i/kJfi6h0e 15:02:38 From Laura to Everyone : I can only see the leave button personally 15:03:00 From Noel Kennedy to Everyone : it works on my mac, i updated zoom this AM 15:03:34 From rturrisi to Everyone : I can see the breakout rooms but I cannot join any 15:42:06 From 59615 to Everyone : thanks 16:20:21 From Maria Liakata to Everyone : I agree with Mark’s comment that we should be working on multi-modal interactions. Could the panel elaborate more on the respective contributions of speech vs language for the detection of different mental health conditions? 16:23:15 From Chloe to Everyone : Do the panelists think that this type of research should be done in tandem with linguistic research? 16:24:31 From Maria Liakata to Everyone : I very much agree on the need for longitudinal modelling. 16:31:37 From Robert Stewart to Everyone : In relation to Heidi’s point, it would be interesting to know how these resources interface with routine clinical care (e.g. quite fragmented service provision in dementia) 16:31:45 From Maria Liakata to Everyone : What would be the panel’s recommendation on automated transcription for mental health. This is something we have definitely struggled with especially for people with dementia. 16:35:06 From rturrisi to Everyone : Why do you think the community of healthcare doesn’t use speech that much? And at the same time, only few researchers groups of ASR work in the biomedical field? 16:37:46 From Robert Stewart to Everyone : Facebook and Google might have a lot of data, but don’t tend to have much access to clinical services (in the UK at least) - or at least any access provided is a lot more controversial and unacceptable to many people. This is where universities do have a potential advantage, as more trusted entities. 16:44:41 From Heidi Christensen to Everyone : True Robert, but recent work, e.g., Google’s Euphonia system (https://sites.research.google/euphonia/about/) shows they are interested in these niche areas. They have a relatively large collection now - sadly no plans to share! 17:02:01 From Maria Liakata to Everyone : Great panel thanks everyone! Got to go. 17:04:22 From Aicha Chorana to Everyone : Thank you all. It was really beneficial. 17:04:44 From Goran Nenadic to Everyone : :) 17:05:07 From Beata Fonferko-Shadrach to Everyone : :) 17:05:28 From Simone Graetzer to Everyone : Thanks to Nick and everyone else in the panel 17:05:29 From Andreas Karwath to Everyone : https://gather.town/i/kJfi6h0e 17:05:32 From PJ to Everyone : Great day - really interesting all the way through 17:05:41 From Heidi Christensen to Everyone : Thank you all - really enjoyed it!! 17:05:49 From laurensrook to Everyone : thank you 17:05:49 From Jaya C to Everyone : Thank you everyone :) 17:05:50 From Paul Rayson to Everyone : Thanks! 17:05:51 From Laura to Everyone : Very informative :) thanks 17:05:56 From Beth Rushton-Woods to Everyone : Thanks all! 17:06:02 From Nicholas Cummins to Everyone : Thanks to everyone for a great day! 17:06:02 From Kristof Anetta to Everyone : Thank you all!