APPLICATION OF RULE-BASED METHOD FOR AUTOMATIC EXTRACTION OF TAGS FROM COLUMN-STYLE PDF-DOCUMENTS

Assel Ospan; Madina Mansurova; Kanat Auyesbay; Talshyn Sarsembayeva; Aman Mussa

doi:10.26577/jpcsit2025336

Авторы

Assel Ospan Al Farabi Kazakh National University, Almaty, Kazakhstan https://orcid.org/0000-0002-1860-6997 (неаутентифицированный)
Madina Mansurova Al Farabi Kazakh National University, Almaty, Kazakhstan https://orcid.org/0000-0002-9680-2758 (неаутентифицированный)
Kanat Auyesbay Al Farabi Kazakh National University, Almaty, Kazakhstan https://orcid.org/0009-0001-3529-9888 (неаутентифицированный)
Talshyn Sarsembayeva Al Farabi Kazakh National University, Almaty, Kazakhstan https://orcid.org/0000-0001-7668-2640 (неаутентифицированный)
Aman Mussa Al Farabi Kazakh National University, Almaty, Kazakhstan https://orcid.org/0009-0001-9972-7677 (неаутентифицированный)

DOI:

https://doi.org/10.26577/jpcsit2025336

Ключевые слова:

PDF parsing, Rule-based extraction, Metadata extraction, Document structure recognition, Text mining, Low-resource language processing, Knowledge base for LLMs

Аннотация

В данном исследовании представлен гибридный пайплайн на основе правил (rule-based) для автоматического извлечения структурированных метаданных из PDF-версий казахскоязычных газетных статей, с фокусом на национальной газете «Egemen Qazaqstan». Основная цель — поддержка разработки машиночитаемой базы знаний для последующего обучения больших языковых моделей (LLM) и создания ИИ-ассистента для задач дата-журналистики в Казахстане. Предложенный пайплайн интегрирует три open-source библиотеки — pdfminer.six, PyMuPDF и pdfplumber — для извлечения ключевых элементов: заголовка, автора, даты, аннотации, основного текста, названия издания и категории. Для оценки качества извлечения мы сравнили результаты автоматического парсинга с вручную размеченными эталонными файлами на основе трёх реальных выпусков газеты. Оценка проводилась по трём дополняющим друг друга метрикам: Precision, Textual Semantic Similarity (TSS) и Holistic Precision, которые позволяют учитывать как точные, так и семантически близкие совпадения. Результаты эксперимента показывают, что большинство тегов — особенно формализованные поля, такие как дата, название издания и категория — достигли идеального значения Holistic Precision (1.00), в то время как более вариативные поля, такие как заголовок, также показали высокий результат (более 0.85). После валидации пайплайн был масштабирован на полный корпус из 2 140 PDF-документов газеты за период с 2017 по март 2025 года, в результате чего было успешно конвертировано 159 135 статей в структурированный формат JSON. Полученный корпус представляет собой фундаментальную базу знаний для казахскоязычных ИИ-систем в области журналистики и медиа-аналитики.

Скачивания

Данные по скачиваниям пока не доступны.

Биографии авторов

Assel Ospan, Al Farabi Kazakh National University, Almaty, Kazakhstan

Assel Ospan is a senior lecturer at the Department of Artificial Intelligence and Big Data, al-Farabi Kazakh National University (Almaty, Kazakhstan, assel.ospan@kaznu.edu.kz). Her research focuses on the development of large language models for the Kazakh language, intelligent information extraction, and knowledge base construction. She actively participates in national AI research initiatives and has authored several publications on NLP and data journalism.
ORCID iD: 0000-0002-1860-6997.
Madina Mansurova, Al Farabi Kazakh National University, Almaty, Kazakhstan

Madina Mansurova is the head of the Department of Artificial Intelligence and Big Data, Professor, al-Farabi Kazakh National University (Almaty, Kazakhstan, madina.mansurova@kaznu.edu.kz). She has been successfully working in higher education and actively contributing to the advancement of new technologies. Prof.Mansurova is the author of more than 100 scientific articles, 10 monographs, 2 textbooks approved by the Ministry of Education and Science of the Republic of Kazakhstan, 5 patents for useful models in the field of automation and control, and over 40 copyrights on intellectual property. Since 2012, she has been the scientific supervisor of grant and program-targeted funding projects of the Ministry of Education and Science of the Republic of Kazakhstan. She has published 108 articles indexed in Scopus and Web of Science, with a Hirsch index of 7 in Scopus and 205 citations..
ORCID iD: 0000-0002-9680-2758
Kanat Auyesbay, Al Farabi Kazakh National University, Almaty, Kazakhstan

Kanat Auyesbay is the Dean of the Faculty of Journalism at Al-Farabi Kazakh National University (Almaty, Kazakhstan, kanat.auyesbay@kaznu.edu.kz). He is a journalist-educator who bridges the fields of media and higher education. Dr. Auesbay holds a Candidate of Philological Sciences degree (equivalent to PhD) and has extensive experience in both media production and academic leadership. As a recipient of the Bolashak International Scholarship, Kanat Auesbay completed a research and teaching internship at the University of East Anglia, UK (Norwich, 2013–2014). He served as Chairman of the State Attestation Commission at the Faculty of Journalism and Political Science of L.N. Gumilyov Eurasian National University (2023–2024). Since 2018, he has been a corresponding member of the Kazakhstan Academy of Pedagogical Sciences and a member of the Educational-Methodical Association under the Republican Educational-Methodical Council (ROƏK) for Journalism and Information (2019–2021). He has also served on the expert commission for training specialists abroad under the Bolashak program and has supervised and reviewed numerous theses and doctoral dissertations in media studies. ORCID iD: 0009-0001-3529-9888
Talshyn Sarsembayeva, Al Farabi Kazakh National University, Almaty, Kazakhstan

Talshyn Sarsembayeva is a a senior lecturer at the Department of Artificial Intelligence and Big Data, al-Farabi Kazakh National University (Almaty, Kazakhstan, talshyn.sagdatbek@kaznu.edu.kz). Her work focuses on the integration of artificial intelligence and data processing tools in journalistic practice. She has contributed to projects involving the structuring of large-scale media archives and the development of AI-assisted systems for Kazakh-language content. ORCID iD: 0000-0001-7668-2640.
Aman Mussa, Al Farabi Kazakh National University, Almaty, Kazakhstan

Aman Mussa is a research assistant at the Department of Artificial Intelligence and Big Data, al-Farabi Kazakh National University (Almaty, Kazakhstan, mussa.aman0519@gmail.com). He is engaged in the development of rule-based and hybrid NLP pipelines, with a focus on Kazakh-language PDF processing. His work supports large-scale knowledge base generation for intelligent assistants in data journalism. ORCID iD: 0009-0001-9972-7677.