Psychologists, psycholinguists, and other researchers using language stimuli have been struggling for more than 30 years with the problem of how to analyze experimental data that contain two crossed random effects (items and participants). The classical analysis of variance does not apply; alternatives have been proposed but have failed to catch on, and a statistically unsatisfactory procedure of using two approximations (known as F 1 and F 2) has become the standard. A simple and elegant solution using mixed model analysis has been available for 15 years, and recent improvements in statistical software have made mixed models analysis widely available. The aim of this article is to increase the use of mixed models by giving a concise practical introduction and by giving clear directions for undertaking the analysis in the most popular statistical packages. The article also introduces the djmixed add-on package for SPSS, which makes entering the models and reporting their results as straightforward as possible.
MULTIFILE
With the proliferation of misinformation on the web, automatic misinformation detection methods are becoming an increasingly important subject of study. Large language models have produced the best results among content-based methods, which rely on the text of the article rather than the metadata or network features. However, finetuning such a model requires significant training data, which has led to the automatic creation of large-scale misinformation detection datasets. In these datasets, articles are not labelled directly. Rather, each news site is labelled for reliability by an established fact-checking organisation and every article is subsequently assigned the corresponding label based on the reliability score of the news source in question. A recent paper has explored the biases present in one such dataset, NELA-GT-2018, and shown that the models are at least partly learning the stylistic and other features of different news sources rather than the features of unreliable news. We confirm a part of their findings. Apart from studying the characteristics and potential biases of the datasets, we also find it important to examine in what way the model architecture influences the results. We therefore explore which text features or combinations of features are learned by models based on contextual word embeddings as opposed to basic bag-of-words models. To elucidate this, we perform extensive error analysis aided by the SHAP post-hoc explanation technique on a debiased portion of the dataset. We validate the explanation technique on our inherently interpretable baseline model.
Developing a framework that integrates Advanced Language Models into the qualitative research process.Qualitative research, vital for understanding complex phenomena, is often limited by labour-intensive data collection, transcription, and analysis processes. This hinders scalability, accessibility, and efficiency in both academic and industry contexts. As a result, insights are often delayed or incomplete, impacting decision-making, policy development, and innovation. The lack of tools to enhance accuracy and reduce human error exacerbates these challenges, particularly for projects requiring large datasets or quick iterations. Addressing these inefficiencies through AI-driven solutions like AIDA can empower researchers, enhance outcomes, and make qualitative research more inclusive, impactful, and efficient.The AIDA project enhances qualitative research by integrating AI technologies to streamline transcription, coding, and analysis processes. This innovation enables researchers to analyse larger datasets with greater efficiency and accuracy, providing faster and more comprehensive insights. By reducing manual effort and human error, AIDA empowers organisations to make informed decisions and implement evidence-based policies more effectively. Its scalability supports diverse societal and industry applications, from healthcare to market research, fostering innovation and addressing complex challenges. Ultimately, AIDA contributes to improving research quality, accessibility, and societal relevance, driving advancements across multiple sectors.
De bereikbaarheid en beschikbaarheid van de ambulancezorg staat onder druk. Een belangrijke ingangsklacht van de mensen die 112 bellen is een kortdurende bewusteloosheid. Als deze bewusteloosheid het gevolg is van een verminderde bloedtoevoer in de hersenen noemen we het syncope. Syncope kan onschuldig of ernstig van aard zijn. De risico-inschatting en besluitvorming bij patiënten met syncope in de ambulancezorg is complex. Ambulanceprofessionals moeten in een kort tijdsbestek en onder hoge druk, met veel onderliggende informatie en onzekerheden risico’s inschatten en besluiten of een patiënt ingestuurd moet worden naar de spoedeisende hulp. Bij twee-derde van de ingestuurde syncope patiënten blijkt het niet ernstig te zijn. Twee HAN lectoraten ontwikkelden praktische en onderbouwde handvatten voor de praktijk (RAAK.PUB05.017 en RAAK.IMP.01.036). Deze zijn sinds juli 2022 onderdeel van de landelijke werkwijze. In vervolg hierop heeft de praktijk de lectoraten gevraagd om te kijken of de inzet van digitale- en informatietechnologie, specifiek generatieve kunstmatige intelligentie (AI) op basis van Large Language Models (LLM), hen nog verder kan ondersteunen bij het inschatten van risico’s en besluiten maken bij patiënten met syncope in de ambulancezorg. Deze KIEM-aanvraag is een proof of concept studie. We onderzoeken in hoeverre generatieve AI op basis van LMM technisch goed tekstbestanden kan analyseren op belangrijke medische- en omgevingsfactoren bij patiënten met een syncope. We kiezen voor een pilot concurrente validatiestudie door kwalitatieve tekstanalyse, in combinatie met aanvullende focusgroepinterviews voor de interpretatie van de uitkomsten. Voor de pilot concurrente validatiestudie gebruiken we tekstbestanden uit de Safe End studie. De eerdere analyse van deze tekstbestanden uit de Safe End studie fungeert als de gouden standaard. Zo wordt de validiteit van de generatieve AI-analyse op basis van LMM vastgesteld. In focusgroepinterviews bespreken we de impact en ethische aspecten van de bevindingen voor de praktijk, wetenschap, onderwijs en de (door)ontwikkeling van beslissingsondersteuningsinstrumenten voor de toekomst.
