Hi, I am Jinfen Li
A researcher in the field of Interpretability of Natural Language Processing.
An Interpretable-NLP Researcher
A Machine Learning Engineer
A Social Media Analyst
About Me
My research interests lie in Natural Language Processing and Explainable AI. I develop NLP algorithms such as multi-emotion recognition, discourse parsing and propaganda detection, and apply them to social science topics. Recently, I focus on improving human-AI trust by incorporating explainability into NLP models.
Education
- Ph.D. in Information Studies, 2021 - Present
- M.S. in Computational Linguistics, 2018 - 2020
Syracuse University, Syracuse, NY, USA
- B.S. in Computer Science, 2014 - 2018
Guangdong University of Technology, Guangzhou, China
Contact
- 📮Email: jli284@syr.edu
Projects
NLP models. I develop NLP algorithms such as multi-emotion recognition, discourse parsing and propaganda detection, and apply them to social science topics.
Interpretable Fake News Detection
Abstract
Social media platforms enable information proliferation, but they also pose challenges for people to distinguish true news from fake ones. Various research efforts are made to facilitate fake news detection, such as the development of AI-based tools. Yet, the inscrutable and opaque working mechanisms of these tools make it difficult for people to trust the detection outcomes. To contribute to AI transparency, our proposed study will design an interpretable artificial intelligence, an interpretable agent that extracts indicative text spans from the input texts as the rationales for model predictions (e.g., the veracity of fake news). Specifically, our proposal will rethink the model structure to meet several desiderata of extractive rationales, as well as extract more fine-grained information apart from raw texts such as semantic and discourse information. Our work generates several impacts. For example, it empowers automatic detection and user awareness of fake news. In addition, the interpretable AI will prove a boon to the AI community and boost progress toward a better ethical, transparent, and accountable AI research agenda.
- script.py
1
2
3
4
5
6
7
8
9
10
11
12
This tool is under developed
- Terminal
Given a news article and a query, our tool identifies the most important sentences in the article that are relevant to the query that indicates why the tool think the article is fake or not.
Interpretable Stance Separation
Abstract
From the early days of natural language processing, researchers have captured the meaning of the text by mapping it into a high-dimensional vector space, wherein human interpretations of semantic similarity are presumed to correlate with distance measures in vector space. However, there is no well-defined gold standard of semantic similarity because it depends upon the cultural frames wherein the similarity is assessed. This presents a particular challenge when modern meanings begin to depart from those present in historical training texts. Such misalignment underlies problems of bias in modern language models and presents challenges for interpretability.
One notable example of misalignment stems from a distinction between stance and semantics. Oppositional stances are commonly found in modern issues with significant cultural divergence, and recent work has shown that many existing and widely available pre-trained models do not effectively distinguish between semantically similar but positionally distinct texts (e.g., "climate change is a challenge" vs. "climate change is a hoax"). In this study, we demonstrate a method for using social groups to tune language models to better distinguish between stance groups without disrupting other aspects of semantics. We propose three approaches, all of which outperform a "vanilla" SBERT model according to several measures we introduce here. In particular, we evaluate our approaches against human judgments and show that our tuned models better reflect human interpretations of semantics. Our method paves the way for the future development of "culturally sensitive" embedding spaces that more closely mirror modern cultural interpretations of semantics.
- script.py
1
2
3
4
5
6
7
8
9
10
11
12
This tool is under developed
- Terminal
The large langauge models mix different stances in each cluster, which we are trying to separate as well as maintain their semantic similarities.
Multi-emotion Recognition Using Multi-EmoBERT and Emotion Analysis in Fake News
Abstract
Emotion recognition techniques are increasingly applied in fake
news veracity or stance detection. While multiple co-existing emotions tend to co-occur in a single news article, most existing fake
news detection has only leveraged single-label emotion recognition mechanisms. In addition, the relationship between the emotion of an article and its stance has not been sufficiently explored.
To address these research gaps, we have developed and applied
a multi-label emotion recognition tool called Multi-EmoBERT in
fake news datasets. The tool delivers state-of-the-art performance
on SemEval2018 Task 1. We apply the tool to identify emotions in
several fake news datasets and examine the relationships between
veracity/stance and emotion. Our work demonstrates the potential for predicting multiple co-existing emotions for fake news and
implications against fake news spread.
- script.py
1
2
3
4
5
6
!pip install multi-emotion
from multi_emotion import multi_emotion
print(multi_emotion.predict(["I am so happy today"]))
- Terminal
[{'text': 'i am so happy today',
'pred_label': 'joy,love,optimism',
'probability': '[{"anger": 0.00022063202050048858},
{"anticipation": 0.007108359131962061},
{"disgust": 0.0006860275752842426},
{"fear": 0.00044393239659257233},
{"joy": 0.9998739957809448},
{"love": 0.8244059085845947},
{"optimism": 0.931083083152771},
{"pessimism": 0.0002464792341925204},
{"sadness": 0.007342423778027296},
{"surprise": 0.001668739365413785},
{"trust": 0.009098367765545845}]'}]
As shown in this Figure, some pairs of positive emotions are not as strongly positively correlated as others and are even negatively correlated. For instance, joy and surprise are weakly positively correlated, and love and surprise are negatively correlated. In our study, we capture the nuance of several emotion correlations using a transferable biaffine weight that captures latent feature correlation and further assists in emotion association.
Neural-based RST Parsing And Analysis In Persuasive Discourse
Abstract
Most of the existing studies of language use in social media content have focused on the surface-level linguistic features (e.g., function words and punctuation marks) and the semantic level aspects (e.g., the topics, sentiment, and emotions) of the comments. The writer's strategies of constructing and connecting text segments have not been widely explored even though this knowledge is expected to shed light on how people reason in online environments. Contributing to this analysis direction for social media studies, we build an openly accessible neural RST parsing system that analyzes discourse relations in an online comment. Our experiments demonstrate that this system achieves comparable performance among all the neural RST parsing systems. To demonstrate the use of this tool in social media analysis, we apply it to identify the discourse relations in persuasive and non-persuasive comments and examine the relationships among the binary discourse tree depth, discourse relations, and the perceived persuasiveness of online comments. Our work demonstrates the potential of analyzing discourse structures of online comments with our system and the implications of these structures for understanding online communications.
- script.py
1
2
3
4
5
6
7
8
9
10
11
12
!pip install rst-parser
from rst_parser import rst_parser
tree_results, dis_results = rst_parser.parse(["The scientific community is making significant progress in understanding climate change. Researchers have collected vast amounts of data on temperature fluctuations, greenhouse gas emissions, and sea-level rise.
This data shows a clear pattern of increasing global temperatures over the past century.
However, there are still debates about the causes and consequences of climate change."])
print(dis_results)
- Terminal
( Root (span 1 4)
( Nucleus (span 1 2) (rel2par span)
( Nucleus (leaf 1) (rel2par span) (text _!The scientific community is making significant progress!_) )
( Satellite (leaf 2) (rel2par Elaboration) (text _!in understanding climate change .!_) )
)
( Satellite (span 3 4) (rel2par Elaboration)
( Nucleus (leaf 3) (rel2par span) (text _!Researchers have collected vast amounts of data on temperature fluctuations , greenhouse gas emissions , and sea-level rise .!_) )
( Satellite (leaf 4) (rel2par Contrast) (text _!This data shows a clear pattern of increasing global temperatures over the past century . However , there are still debates about the causes and consequences of climate change .!_) )
)
)
The span represents the cover range of a sequence of EDUs; while the nuclearity status is the semantic role in a relation (i.e., nucleus or satellite). A text tagged by nucleus is more essential than the satellite. Conventionally, there are three nuclearity types including Nucleus-Satellite (NS), Satellite-Nucleus (SN) and Nucleus-Nucleus (NN). As for relation, there are mainly two types of relations: the mono-nuclear relation, such as “Attribution” or “Summary”, with the nuclearity type being either “NS” or “SN”; and the multi-nuclear relation, such as “Contrast” or “Same-Unit”, with the nuclearity types of “NN”.
BERT-based Models Design For Propagandistic Technique and Span Detection
Abstract
This paper describes the BERT-based models proposed for two subtasks in SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. We first build the model for Span Identification (SI) based on SpanBERT, and facilitate the detection by a deeper model and a sentence-level representation. We then develop a hybrid model for the Technique Classification (TC). The hybrid model is composed of three submodels including two BERT models with different training methods, and a feature-based Logistic Regression model. We endeavor to deal with imbalanced dataset by adjusting cost function.
We are in the seventh place in SI subtask (0.4711 of F1-measure), and in the third place in TC subtask (0.6783 of F1-measure) on the development set.
- script.py
1
2
3
4
5
6
7
8
The pip-installable tool is under-developed,
but the source code is available.
- Terminal
In this paper, we develop two systems for two subtasks in the SemEval-2020 Task 11: (1) Span Identification (SI) (2) Technique Classification (TC) in News Articles respectively. The SI subtask focus on identifying fragments in a given plain-text document which contain at least one propaganda technique; while TC subtask aims to classify the applied propaganda technique given a propagandistic text fragment.
Empathetic Chatbot
Abstract
To explore the potential of using AI-based chatbots for religious activities, we developed Verse22, a GPT-2 based chatbot that exchanges Bible verses with a user and provides related information.
- script.py
1
2
3
4
5
6
7
8
The pip-installable tool is under-developed,
and the source code is not available yet.
- Terminal
Chat screenshot with Verse22 about the experience sharing module
all projects
current projects
past projects
Publications
Conference, Workshop papers
Teachings
NLP, Applied Deep Learning .
Instructor: Lu Xiao TA: Jinfen Li Semester: Fall, 2021
Chatbot
Instructor: Acuna Daniel TA: Jinfen Li Semester: Spring, 2022
CNN
Notebooks
I am working on two notebooks.
Let's make NLP interpretable!
Interpretability opens an opportunity to understand the model and its predictions. It is a crucial step in the model development process. It also paves the way for sustainable and ethical AI.
