Question Solved

Is a two-year gap between bachelor's and master's degree acceptable for PhD applications abroad?

Back to Forum
Starting 0 pts 0 followers
Bearys Institute of Technology · Posted
Materials Science

Background

I am a second-year PhD student working on multilingual NLP at NITK. My research involves building named entity recognition systems for code-mixed Telugu-English text, but I am struggling with the fine-tuning process for large language models.

The problem

I have around 8,000 annotated sentences, which I know is relatively small for fine-tuning something like XLM-R. When I fine-tune with default settings, I get reasonable results on validation but the model overfits badly by epoch 5.

What I have tried

  • Reduced learning rate to 1e-5
  • Added dropout at 0.3
  • Early stopping with patience=3
  • Tried LoRA fine-tuning as an alternative

Has anyone dealt with similar low-resource scenarios? Is LoRA actually recommended for NER tasks, or is full fine-tuning with heavy regularization the better path?

Any references to papers or GitHub repos with similar setups would be very helpful.

Sign in to interact

Sign in to join the discussion.

10 Replies

Oldest Top
0
Pooja Nambiar Growing 115 pts · Accepted answer

The ethics review situation in Indian universities is actually more complex than most people realize.

There is no uniform national framework equivalent to the US Common Rule. Each institution has its own Institutional Ethics Committee (IEC) or Institutional Review Board (IRB).

For anonymous survey research with no identifiable data and no vulnerable populations: most Indian IECs would classify this as exempt, but 'exempt' still needs a formal determination from the IEC.it is not self-certifying.

My practical advice: go to your institution's research ethics office now, describe what you did, and request an exempt determination in writing. Better to have documentation than to discover the issue during thesis review.

0
Ashwin Murthy Distinguished 560 pts · Accepted answer

I am in a very similar situation. Would you be willing to share the outline of your PMRF proposal? Not the content.just the section headings and approximate word allocation.

0
Fatima Sheikh Growing 95 pts · Accepted answer

+1 to everything said above. My experience was identical.

0
Tara Singh Starting 5 pts · Accepted answer

I found that writing a paper abstract first (before the paper) and then reverse-engineering the paper from the abstract helped with clarity enormously.

0
Revathi Chandrasekaran Active 410 pts · Accepted answer

I found that writing a paper abstract first (before the paper) and then reverse-engineering the paper from the abstract helped with clarity enormously.

0
Rekha Shetty Starting 40 pts · Accepted answer

This is exactly what I needed to hear. One clarifying question: you mentioned FedProx handles non-IID data better.does that mean I should switch to FedProx by default, or only if I observe poor convergence with FedAvg first?

0
Bhavana Reddy Starting 30 pts · Accepted answer

This is a known issue with XLM-R on code-mixed data. The MuRIL preprint has a section specifically comparing these.

0
Karthik Rajan Growing 180 pts · Accepted answer

For longitudinal data with missing values, mixed-effects models handle this more gracefully than imputation in most cases.

0
Indira Balan · Accepted answer

This is exactly what I needed to hear. One clarifying question: you mentioned FedProx handles non-IID data better.does that mean I should switch to FedProx by default, or only if I observe poor convergence with FedAvg first?

0
Vikram Bhatia Active 420 pts · Accepted answer

Check whether your institution has a Springer/Elsevier waiver agreement.many NITs and IITs do.