Question Solved

What should I know about filing an Indian patent as a student? (52)

Back to Forum
Active 480 pts 0 followers
University of Mysore · Posted
Curriculum & Pedagogy Public Health & Epidemiology Earth & Climate Science

Background

I am a second-year PhD student working on multilingual NLP at NITK. My research involves building named entity recognition systems for code-mixed Telugu-English text, but I am struggling with the fine-tuning process for large language models.

The problem

I have around 8,000 annotated sentences, which I know is relatively small for fine-tuning something like XLM-R. When I fine-tune with default settings, I get reasonable results on validation but the model overfits badly by epoch 5.

What I have tried

  • Reduced learning rate to 1e-5
  • Added dropout at 0.3
  • Early stopping with patience=3
  • Tried LoRA fine-tuning as an alternative

Has anyone dealt with similar low-resource scenarios? Is LoRA actually recommended for NER tasks, or is full fine-tuning with heavy regularization the better path?

Any references to papers or GitHub repos with similar setups would be very helpful.

Sign in to interact

Sign in to join the discussion.

4 Replies

Oldest Top
0
Ashwin Murthy Distinguished 560 pts · Accepted answer

Great question.I went through something very similar in my second year.

The key insight for me was that LoRA is actually quite well-suited for NER tasks, especially in low-resource settings. I would recommend:

  1. Use LoRA with r=8 or r=16.don't go higher for 8K samples
  2. Apply LoRA to attention layers only, not the feed-forward layers
  3. Use a cosine learning rate schedule with warm-up (10% of steps)

For Telugu-English code-mixed NER specifically, you might also look at MuRIL.it is pretrained on Indian language data and often outperforms XLM-R on Indic tasks even with less fine-tuning data.

0
Replying to Ashwin Murthy
Rajesh Kumar · Accepted answer

Just to make sure I understand the Zotero workflow correctly.is the .bib file synced to Overleaf automatically every time you add a new paper, or do you need to trigger an export manually?

0
Replying to Ashwin Murthy
Rohan Desai Distinguished 950 pts · Accepted answer

This is a known issue with XLM-R on code-mixed data. The MuRIL preprint has a section specifically comparing these.

0
Karthik Rajan Growing 180 pts · Accepted answer

I have successfully received PMRF (lateral entry, second year). Happy to share what worked in my proposal.

On the research proposal:

  • 60–70% technical approach, 30% context and impact
  • Preliminary results absolutely help. Coursework projects are fine if genuinely relevant
  • The societal impact section should be substantive.PMRF is a national initiative; reviewers want to see why this matters

On citations: Yes, cite your MTech thesis. Framing your PhD as an extension of prior work you did shows continuity of thought.