Tech

Beyond IceBERT: Open-Data ModernBERT Encoders for Icelandic from 12M to 1B Parameters

By Hafsteinn Einarsson · arXiv cs.CL

cs.CL

arXiv:2610.36230v1 Announce Type: new Abstract: IceBERT, the reference encoder for Icelandic, found that a cleaned web crawl is enough to train a strong model and that starting from a multilingual model can help on some tasks. We…

Published: October 28, 2026 (1 h ago)
Outlet: arXiv cs.CL
Section: Tech
Length here: Headline and 37-word snippet. The full article is at the source.

More from arXiv cs.CL and around the same time

An open laptop and a stack of books on a desk

3, 7, 14, 17, 42: Instruction Tuning Installs and Sharpens Number-Choice Heuristics in Language ModelsarXiv:2610.36178v1 Announce Type: new Abstract: We measure, without sampling error, the probability a language model assigns to each integer from 1 to N when asked for a random number. For a fixed prompt the model defines one…arXiv cs.CL · 1 h ago · reads at arXiv cs.CL

AK_QMUL at ArGuard Shared Tasks: A Single Fine-Tuned JAIS for Arabic Harmful-Prompt DetectionarXiv:2610.36360v1 Announce Type: new Abstract: We present AK_QMUL's submissions to the ArGuard-2026 shared tasks on Arabic harmful-prompt detection: a binary safe/unsafe task (B1) and a seven-way harm-domain task (B2). Both…arXiv cs.CL · 1 h ago · reads at arXiv cs.CL

Benchmarking Task Takeover Across Agent HandoffsarXiv:2610.36523v1 Announce Type: new Abstract: As agents collaborate on longer tasks, responsibility may pass between models before the work is complete. The receiving model must build on prior decisions, observations, and…arXiv cs.CL · 1 h ago · reads at arXiv cs.CL

CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement LearningarXiv:2610.36437v1 Announce Type: new Abstract: When a Large Language Model (LLM) is trained with Reinforcement Learning (RL) on multiple tasks, a task's primary reward can gate auxiliary rewards such as a length reward: on a…arXiv cs.CL · 1 h ago · reads at arXiv cs.CL

Detect Before You Disambiguate: Perturbation-Based Input Ambiguity Detection in LLMsarXiv:2610.36353v1 Announce Type: new Abstract: Large language models (LLMs) often answer ambiguous inputs as if they had a single intended meaning. This is problematic because input ambiguity is an upstream source of…arXiv cs.CL · 1 h ago · reads at arXiv cs.CL

Do AI Scientists Know What's New?arXiv:2610.36511v1 Announce Type: new Abstract: A major challenge facing AI scientific discovery pipelines is how to determine when a generated idea is meaningfully novel. Ideally, novelty checks in these pipelines must be able…arXiv cs.CL · 1 h ago · reads at arXiv cs.CL

This page holds the headline and a short snippet as published by arXiv cs.CL, whose feed they came from. The story itself, the photography and the rights stay with them, and the button above goes straight to their site. We keep this page so the headline has a stable address and a record of when it ran.