Question Solved

What tools do people use for collaborative annotation of research datasets?

Back to Forum
Starting 0 pts 0 followers
R.V. College of Engineering · Posted
Neuroscience

Context

I want to start using federated learning for my research on privacy-preserving healthcare analytics. The datasets involve patient records from three hospitals and we cannot share data across sites.

What I am looking for:

  • Python frameworks that are actively maintained and used in research
  • Any frameworks with good support for heterogeneous data distributions (non-IID)
  • Something that has a reasonably gentle learning curve for a researcher (not a systems engineer)

I have looked at PySyft and Flower. Any experience with either, or other alternatives I should consider?

Sign in to interact

Sign in to join the discussion.

38 Replies

Oldest Top
0
Disha Malhotra Starting 8 pts · Accepted answer

Could you clarify what you mean by results-blind review? I have heard of it but never understood how it works in practice. Wouldn't reviewers need to see results to evaluate whether the methodology is sound?

0
Replying to Disha Malhotra
test tester Growing 165 pts · Accepted answer

I am in a very similar situation. Would you be willing to share the outline of your PMRF proposal? Not the content.just the section headings and approximate word allocation.

0
Pooja Nambiar Growing 115 pts · Accepted answer

As a senior faculty member (25 years in the system), I want to offer a different perspective. Yes, the pressures are real. But I have also seen the other side: students who produce excellent work, publish honestly, and build reputations over time. The system is imperfect but not entirely broken.

What has actually helped my students: choosing venues carefully (a few strong papers rather than many weak ones), developing genuine collaborations rather than transactional coauthorships.

0
Divya Krishnan Growing 85 pts · Accepted answer

The ethics review situation in Indian universities is actually more complex than most people realize.

There is no uniform national framework equivalent to the US Common Rule. Each institution has its own Institutional Ethics Committee (IEC) or Institutional Review Board (IRB).

For anonymous survey research with no identifiable data and no vulnerable populations: most Indian IECs would classify this as exempt, but 'exempt' still needs a formal determination from the IEC.it is not self-certifying.

My practical advice: go to your institution's research ethics office now, describe what you did, and request an exempt determination in writing. Better to have documentation than to discover the issue during thesis review.

0
Sunil Bhattacharya Active 390 pts · Accepted answer

Highly recommend checking out the PMRF portal for the actual numbers.they update the stipend structure annually.

0
Wajid Khan Growing 110 pts · Accepted answer

I am in a very similar situation. Would you be willing to share the outline of your PMRF proposal? Not the content.just the section headings and approximate word allocation.

0
Chirag Mehta Starting 15 pts · Accepted answer

Just to make sure I understand the Zotero workflow correctly.is the .bib file synced to Overleaf automatically every time you add a new paper, or do you need to trigger an export manually?

0
Arjun Venkatesan Growing 195 pts · Accepted answer

If you are not already using a reading group format with your students, I would strongly recommend it. Forces everyone to articulate what they read.

0
Neha Agarwal Growing 90 pts · Accepted answer

Flower (flwr) is the most actively maintained and research-friendly federated learning framework right now. My group has been using it for 18 months on a healthcare project with 4 hospital sites.

Strengths:

  • Clean simulation API that lets you test on a single machine before deploying
  • Good support for custom aggregation strategies (FedProx, FedNova)
  • Active Discord community with quick responses from the maintainers

For non-IID data specifically, look at the FedProx strategy.it handles heterogeneous data distributions much better than FedAvg.

0
Bhavana Reddy Starting 30 pts · Accepted answer

I am in a very similar situation. Would you be willing to share the outline of your PMRF proposal? Not the content.just the section headings and approximate word allocation.

0
Asha Pillai Distinguished 880 pts · Accepted answer

Respectfully, I think this framing misses the systemic dimension. Individual researchers cannot solve a collective action problem. If you submit to open-access journals and your colleague submits to Nature, your colleague gets the promotion. Until evaluation criteria change at the institutional level.which requires policy intervention.individual choices have minimal impact.

0
Quamar Ahmed Growing 65 pts · Accepted answer

Thank you for this detailed answer! A quick follow-up: when you mention cosine learning rate with warm-up, are you warming up the LoRA parameters specifically or the entire model including the frozen backbone?

0
Lakshmi Devi Growing 75 pts · Accepted answer

Just to make sure I understand the Zotero workflow correctly.is the .bib file synced to Overleaf automatically every time you add a new paper, or do you need to trigger an export manually?

0
Replying to Lakshmi Devi
Arjun Venkatesan Growing 195 pts · Accepted answer

Thank you for the honest take. This is the kind of answer I was looking for.not the sanitized version.

0
Replying to Lakshmi Devi
Sunil Bhattacharya Active 390 pts · Accepted answer

Thank you for the honest take. This is the kind of answer I was looking for.not the sanitized version.

0
Replying to Lakshmi Devi
Deepa Krishnamurthy Distinguished 810 pts · Accepted answer

The Fulbright SoP is very different from a typical PhD SoP. It needs to focus on cultural exchange and diplomacy, not just research.

0
Pooja Nambiar Growing 115 pts · Accepted answer

Thank you for this detailed answer! A quick follow-up: when you mention cosine learning rate with warm-up, are you warming up the LoRA parameters specifically or the entire model including the frozen backbone?

0
Rajesh Kumar · Accepted answer

For Zotero vs BibTeX management, I moved to Zotero + Better BibTeX 3 years ago and have not looked back.

My workflow:

  1. Zotero browser connector captures papers instantly from ArXiv, Google Scholar, ACM DL, IEEE Xplore
  2. Better BibTeX generates a clean .bib file that auto-updates whenever I add a paper
  3. Overleaf directly pulls from the .bib file via Dropbox sync

One tip: create a Zotero collection per paper/project. When you export the .bib, export only that collection to keep it clean.

0
Varsha Pillai Growing 60 pts · Accepted answer

I can answer your question about IEEE Transactions review timelines from my experience submitting to IEEE TNNLS.

For Q1 IEEE journals, expect:

  • Initial desk review: 2–4 weeks
  • Full peer review: 3–6 months
  • Major revision decision: another 2–3 months for re-review

As for code and data, most IEEE journals now request code upon acceptance, not submission. Regarding your 2.3% improvement.that is borderline. You will need very strong experimental rigor: multiple runs, confidence intervals, and significance tests.

0
Replying to Varsha Pillai
Geeta Rao · Accepted answer

This is very reassuring. I was worried the null result would kill the paper but your framing around transparency and effect sizes makes a lot of sense.

0
Replying to Varsha Pillai
Tara Singh Starting 5 pts · Accepted answer

Just to make sure I understand the Zotero workflow correctly.is the .bib file synced to Overleaf automatically every time you add a new paper, or do you need to trigger an export manually?

0
Anita Rao · Accepted answer

This is exactly what I needed to hear. One clarifying question: you mentioned FedProx handles non-IID data better.does that mean I should switch to FedProx by default, or only if I observe poor convergence with FedAvg first?

0
Xena D'Souza Starting 25 pts · Accepted answer

CSIR-UGC NET preparation: past papers are the most important resource. Everything else is secondary.

0
Lakshmi Devi Growing 75 pts · Accepted answer

Strongly agree with the point about choosing venues carefully. One paper in NeurIPS > five papers in obscure workshops.

0
Kavitha Subramanian Active 365 pts · Accepted answer

The ethics review situation in Indian universities is actually more complex than most people realize.

There is no uniform national framework equivalent to the US Common Rule. Each institution has its own Institutional Ethics Committee (IEC) or Institutional Review Board (IRB).

For anonymous survey research with no identifiable data and no vulnerable populations: most Indian IECs would classify this as exempt, but 'exempt' still needs a formal determination from the IEC.it is not self-certifying.

My practical advice: go to your institution's research ethics office now, describe what you did, and request an exempt determination in writing. Better to have documentation than to discover the issue during thesis review.

0
Hemant Patwa · Accepted answer

Open access is the right direction but the APCs are prohibitively expensive for many Indian researchers without institutional funding.

0
Hema Suresh Starting 20 pts · Accepted answer

Thank you for this detailed answer! A quick follow-up: when you mention cosine learning rate with warm-up, are you warming up the LoRA parameters specifically or the entire model including the frozen backbone?

0
Replying to Hema Suresh
Nandita Ghosh Distinguished 720 pts · Accepted answer

Thank you for this detailed answer! A quick follow-up: when you mention cosine learning rate with warm-up, are you warming up the LoRA parameters specifically or the entire model including the frozen backbone?

0
Replying to Hema Suresh
Farhan Mirza · Accepted answer

This is a known issue with XLM-R on code-mixed data. The MuRIL preprint has a section specifically comparing these.

0
Bhavana Reddy Starting 30 pts · Accepted answer

I have successfully received PMRF (lateral entry, second year). Happy to share what worked in my proposal.

On the research proposal:

  • 60–70% technical approach, 30% context and impact
  • Preliminary results absolutely help. Coursework projects are fine if genuinely relevant
  • The societal impact section should be substantive.PMRF is a national initiative; reviewers want to see why this matters

On citations: Yes, cite your MTech thesis. Framing your PhD as an extension of prior work you did shows continuity of thought.

0
Replying to Bhavana Reddy
Priya Nair · Accepted answer

Could you clarify what you mean by results-blind review? I have heard of it but never understood how it works in practice. Wouldn't reviewers need to see results to evaluate whether the methodology is sound?

0
Replying to Bhavana Reddy
Suresh Iyer · Accepted answer

If you are not already using a reading group format with your students, I would strongly recommend it. Forces everyone to articulate what they read.

0
Replying to Bhavana Reddy
Admin NITK · Accepted answer

The IEEE TPAMI turnaround in my case was 7 months for the first review. Plan accordingly.

0
Rajesh Kumar · Accepted answer

Overleaf + GitHub integration is underrated. You get version control and collaboration in one place.

0
Replying to Rajesh Kumar
Manish Tiwari Starting 10 pts · Accepted answer

+1 to everything said above. My experience was identical.

0
Replying to Rajesh Kumar
Bhavana Reddy Starting 30 pts · Accepted answer

Check whether your institution has a Springer/Elsevier waiver agreement.many NITs and IITs do.

0
Replying to Rajesh Kumar
Rekha Shetty Starting 40 pts · Accepted answer

When you say 'multiple runs with confidence intervals', do you mean running the full training pipeline multiple times with different random seeds and reporting mean ± std? Or something more statistically rigorous?

0
Preethi Anand Distinguished 690 pts · Accepted answer

Flower (flwr) is the most actively maintained and research-friendly federated learning framework right now. My group has been using it for 18 months on a healthcare project with 4 hospital sites.

Strengths:

  • Clean simulation API that lets you test on a single machine before deploying
  • Good support for custom aggregation strategies (FedProx, FedNova)
  • Active Discord community with quick responses from the maintainers

For non-IID data specifically, look at the FedProx strategy.it handles heterogeneous data distributions much better than FedAvg.