Question Solved

How do I cite an arXiv preprint that has since been published in a journal?

Back to Forum
Starting 0 pts 0 followers
Indian Institute of Science · Posted
Robotics & Automation Economics Astronomy & Astrophysics

Background

I am a second-year PhD student working on multilingual NLP at NITK. My research involves building named entity recognition systems for code-mixed Telugu-English text, but I am struggling with the fine-tuning process for large language models.

The problem

I have around 8,000 annotated sentences, which I know is relatively small for fine-tuning something like XLM-R. When I fine-tune with default settings, I get reasonable results on validation but the model overfits badly by epoch 5.

What I have tried

  • Reduced learning rate to 1e-5
  • Added dropout at 0.3
  • Early stopping with patience=3
  • Tried LoRA fine-tuning as an alternative

Has anyone dealt with similar low-resource scenarios? Is LoRA actually recommended for NER tasks, or is full fine-tuning with heavy regularization the better path?

Any references to papers or GitHub repos with similar setups would be very helpful.

Sign in to interact

Sign in to join the discussion.

9 Replies

Oldest Top
0
Om Prakash Growing 55 pts · Accepted answer

I'd push back slightly on the idea that industry collaboration is universally beneficial for academic research. The incentive misalignment is real.

Industry partners want deliverables on a schedule. Academic research is often exploratory and unpredictable. When the two clash, it is usually the researcher who has to compromise.either by rushing results, constraining publication, or steering away from results that are inconvenient for the sponsor.

This doesn't mean industry collaboration is bad. But the terms matter enormously. IP rights, publication rights, and the ability to pursue negative results should be negotiated upfront and protected in writing.

0
Indira Balan · Accepted answer

I think you are conflating two separate problems here. The publish-or-perish culture creates pressure, yes.but the solution is not to criticize researchers who are navigating an unfair system. The real problem is with evaluation committees who treat publication count as a proxy for research quality. Fix the evaluation, and the incentives change.

Also worth noting: open review has been implemented in some fields and has its own problems. It can disadvantage early-career researchers who fear retaliation from senior colleagues they have reviewed critically.

0
Replying to Indira Balan
Sunil Bhattacharya Active 390 pts · Accepted answer

I found that writing a paper abstract first (before the paper) and then reverse-engineering the paper from the abstract helped with clarity enormously.

0
Quamar Ahmed Growing 65 pts · Accepted answer

For longitudinal data with missing values, mixed-effects models handle this more gracefully than imputation in most cases.

0
Ekta Choudhary Starting 45 pts · Accepted answer

I have successfully received PMRF (lateral entry, second year). Happy to share what worked in my proposal.

On the research proposal:

  • 60–70% technical approach, 30% context and impact
  • Preliminary results absolutely help. Coursework projects are fine if genuinely relevant
  • The societal impact section should be substantive.PMRF is a national initiative; reviewers want to see why this matters

On citations: Yes, cite your MTech thesis. Framing your PhD as an extension of prior work you did shows continuity of thought.

0
Replying to Ekta Choudhary
Chirag Mehta Starting 15 pts · Accepted answer

I am in a very similar situation. Would you be willing to share the outline of your PMRF proposal? Not the content.just the section headings and approximate word allocation.

0
Replying to Ekta Choudhary
Chirag Mehta Starting 15 pts · Accepted answer

Overleaf + GitHub integration is underrated. You get version control and collaboration in one place.

0
Farhan Mirza · Accepted answer

+1 to everything said above. My experience was identical.

0
Neha Agarwal Growing 90 pts · Accepted answer

Flower (flwr) is the most actively maintained and research-friendly federated learning framework right now. My group has been using it for 18 months on a healthcare project with 4 hospital sites.

Strengths:

  • Clean simulation API that lets you test on a single machine before deploying
  • Good support for custom aggregation strategies (FedProx, FedNova)
  • Active Discord community with quick responses from the maintainers

For non-IID data specifically, look at the FedProx strategy.it handles heterogeneous data distributions much better than FedAvg.