Skip to main content
literica[ai]
FEATURES · THE WORKSPACE

Everything you do
with papers, in one place.

Read, search, ask, synthesize, collaborate — all grounded in your library. Every claim traceable to a page.

/02 · COMPREHENSION

Reads what other
AIs skip.

Formulas. Tables. Figures. Every layer.

READING · 1 page · 412 tokens · 1 equation · 1 table · 1 figure
EQUATION 4.2 · UNDERSTOOD

Policy-gradient objective

Maximizes expected reward by adjusting policy parameters θ in the direction of higher-advantage actions. Variance-reduced via the advantage function Aπ in place of raw returns.

type objectivefamily policy gradientcf. REINFORCE, A2C
TABLE 2 · PARSED

Best: Ours-L  87.4% top-1 @ 124M params

Linear accuracy gain with model size; diminishing returns past 56M. 7× FLOPs cost from S→L for +3.2 pts top-1.

rows 4cols 5best row Ours-Lvs. baseline +9.3 pt
FIGURE 3 · CAPTIONED

Attention concentrates on tokens 5–6 early, disperses by layer 4

Heatmap shows sharp peaks in layers 1–2 for content tokens, with broadening receptive field by layer 4. Consistent with finding in §4.2.

peak L2 · tok 5–6spread increasing w/ depth
/03 · LITERATURE REVIEW

A draft review,
written from your papers.

Pick a folder. Get a structured review, every claim cited.

← ReviewsAttention mechanisms in modern transformer architecturesGenerating✓ Completed·3 sources·~312 words↓ Export

Attention mechanisms in modern transformer architectures

Introduction

Attention reshaped how sequence models capture long-range dependencies3. The field consolidated around self-attention after1 showed recurrence could be discarded; pretraining work2 then proved attention layers transfer across NLP tasks.

Themes

Three themes emerge. Architectural minimalism1 prioritizes parallelism. Bidirectional context2 reframes pretraining as denoising. Soft alignment3 anticipated both, reframing translation as attention-weighted lookup.

Methods

All three works rely on dot-product or additive attention13, scale to large parallel training, and report ablations on heads and alignment depth. Evaluation centers on translation BLEU13 and downstream NLP benchmarks2.

Gaps

Two gaps stand out. The corpus is silent on the compute–quality trade-off — none report wall-clock or energy budgets. Second, the analysis of failure modes (attention sinks, head collapse) has not yet entered this slice of the literature.

Conclusion

The arc runs from soft alignment as an addition to RNNs3, to attention as the only mechanism1, to attention as substrate for transferable pretraining2. Future reviews should incorporate efficiency- and analysis-oriented work.

/04 · CITATION NETWORK

See how your sources connect.

Visualize how papers in your library reference each other. Seminal works pull to the center; clusters self-organize by topic; outliers float free. Hover any node to see what it cites and what cites it.

Seminal
Cluster member
Outlier
Attention Is All You NeedBERTGPT-2Scaling LawsT5
/05 · LENS

Search beyond
your library.

Semantic Scholar, Crossref, OpenAlex, arXiv — searched in parallel, deduped, reranked. Ask why a result matters. Save, cite, or chat with the abstract.

Lens · External Search
attention rollout in transformer encodersSearch ↵
AGGREGATED FROMSemantic Scholar· 47Crossref· 31OpenAlex· 28arXiv· 19125 results · reranked
[1]

Sparse Mixture-of-Experts for Long-Context Transformers

rel 0.94
Patel, R., Chen, L., Okafor, M.·NeurIPS · 2024·412 citations·Semantic Scholar

We propose a sparse routing scheme that activates only k of n expert blocks per token, yielding near-linear scaling…

WHY RELEVANTBuilds directly on the attention-rollout method you cited in §4. Reports the same accuracy plateau past 56M params.
[2]

Attention Sinks and Head Collapse in Long Sequences

rel 0.91
Devlin, S., Karpov, A.·arXiv:2403.12104·87 citations·arXiv

Empirical analysis of failure modes in extended-context decoders; introduces a diagnostic for measuring attention dispersal across…

[3]

Efficient Inference for Encoder–Decoder Stacks at Web Scale

rel 0.83
Yamada, K. et al.·ACL · 2023·156 citations·Crossref

Production deployment notes on a 12-billion-parameter encoder–decoder serving 9 languages, with latency budgets and…

/06 · WORKSPACE

Write, plan, ship —
together.

Real-time co-authoring. Kanban for the reading pipeline. Reusable workflows that batch-extract across a folder.

[doc]

Documents

Co-author in real time.

H1 · H2 · B · IAMK

Attention rollout — meeting notes

The encoder layers concentrate on tokens 5–6 by layer 2, then disperse by layer 4 — see Vaswani §3.

Open question: does this still hold past 8k context?

M
Marcus · 2m
Worth checking the long-context eval from Lincoff ’23.
[kan]

Kanban

Manage the reading pipeline.

To read3
Vaswani — Attention Is All You Need
📄 pdf
BERT pretraining (Devlin)
📄 pdf
Sparse MoE (Patel)
Reading1
Long-context eval (Lincoff)
★ key
Synthesized1
Attention rollout (Aktay)
✓ cited
+ card⌘ template: lit-review pipeline
[flow]

Workflows

Run a prompt chain across a folder.

METHODOLOGY EXTRACTOR · v2batch · 24 docs
  1. 1
    Extract methodology
    ctx · § Methods
    ready
  2. 2
    List contributions
    ctx · Full text
    ready
  3. 3
    Pull experimental results
    ctx · Tables only
    running
  4. 4
    Summarize for non-experts
    ctx · Abstract + concl.
    pending
3 / 4 blocks
68%
/07 · INTEGRATIONS

Bring papers in from where they live.

Connect Zotero or Google Drive and your papers flow into Literica AI — no re-uploading, no duplicate libraries. Or drag PDFs straight in.

[zot]

Zotero

Connect your account and pick the collections you want indexed. New papers added in Zotero appear in Literica AI automatically.

[gd]

Google Drive

Pick PDFs from any Drive folder you already organize papers in. Updates sync when files change.

[pdf]

Drag & drop

Or just drop PDFs into the workspace. Scanned, multi-column, equation-heavy — all parsed the same way.

See it on your own
library.

Free to start. No credit card. Your library stays yours — we don't train on it.