Question Solved

Is it okay to use ChatGPT for grammar-checking my research paper? (87)

Back to Forum
Growing 50 pts 0 followers
Alva's Institute of Engineering and Technology · Posted
Higher Education Policy Environmental Science

Background

I am a second-year PhD student working on multilingual NLP at NITK. My research involves building named entity recognition systems for code-mixed Telugu-English text, but I am struggling with the fine-tuning process for large language models.

The problem

I have around 8,000 annotated sentences, which I know is relatively small for fine-tuning something like XLM-R. When I fine-tune with default settings, I get reasonable results on validation but the model overfits badly by epoch 5.

What I have tried

  • Reduced learning rate to 1e-5
  • Added dropout at 0.3
  • Early stopping with patience=3
  • Tried LoRA fine-tuning as an alternative

Has anyone dealt with similar low-resource scenarios? Is LoRA actually recommended for NER tasks, or is full fine-tuning with heavy regularization the better path?

Any references to papers or GitHub repos with similar setups would be very helpful.

Sign in to interact

Sign in to join the discussion.

9 Replies

Oldest Top
0
Ekta Choudhary Starting 45 pts · Accepted answer

This is exactly what I needed to hear. One clarifying question: you mentioned FedProx handles non-IID data better.does that mean I should switch to FedProx by default, or only if I observe poor convergence with FedAvg first?

0
Replying to Ekta Choudhary
Admin NITK · Accepted answer

The Fulbright SoP is very different from a typical PhD SoP. It needs to focus on cultural exchange and diplomacy, not just research.

0
Replying to Ekta Choudhary
Nandita Ghosh Distinguished 720 pts · Accepted answer

Highly recommend checking out the PMRF portal for the actual numbers.they update the stipend structure annually.

0
Jaya Lakshmanan Growing 50 pts · Accepted answer

Strongly agree with the point about choosing venues carefully. One paper in NeurIPS > five papers in obscure workshops.

0
Replying to Jaya Lakshmanan
Hemant Patwa · Accepted answer

Highly recommend checking out the PMRF portal for the actual numbers.they update the stipend structure annually.

0
Replying to Jaya Lakshmanan
Xena D'Souza Starting 25 pts · Accepted answer

Just to make sure I understand the Zotero workflow correctly.is the .bib file synced to Overleaf automatically every time you add a new paper, or do you need to trigger an export manually?

0
Replying to Jaya Lakshmanan
Jagdish Rawat · Accepted answer

+1 to everything said above. My experience was identical.

0
Chirag Mehta Starting 15 pts · Accepted answer

This is a known issue with XLM-R on code-mixed data. The MuRIL preprint has a section specifically comparing these.

0
Xena D'Souza Starting 25 pts · Accepted answer

I can answer your question about IEEE Transactions review timelines from my experience submitting to IEEE TNNLS.

For Q1 IEEE journals, expect:

  • Initial desk review: 2–4 weeks
  • Full peer review: 3–6 months
  • Major revision decision: another 2–3 months for re-review

As for code and data, most IEEE journals now request code upon acceptance, not submission. Regarding your 2.3% improvement.that is borderline. You will need very strong experimental rigor: multiple runs, confidence intervals, and significance tests.