Rotem Israeli – Research Engineer

Hugging Face

GitHub

irotem98@gmail.com

(+972) 53-432-6592

Professional Experience

Multimodal AI Research Engineer at Idomoo

Apr 2025 – Present

Built multimodal training and evaluation pipelines for code-generation and screenshot-to-webpage models using LoRA, vLLM, structured-output validation, and LLM-as-judge evaluation.
Developed reward-modeling and GRPO-style RL workflows for webpage aesthetics and layout quality.
Built and optimized production inference with vLLM and vLLM-Omni across LLM, image, and video workloads.

Conversational AI Engineer at NLPearl

Real-time pause detection & starter suggestions via fine-tuned LLMs.
Explored encoder vs decoder architectures with LoRA + multi-stage training.
Designed an SLM that generates task-specific tokens for multi-task inference.

Machine Learning Engineer at Israeli Navy

Adapted vision and audio models for sonar and signal-processing tasks, including EnCodec/WavTokenizer-style representations.
Trained self-supervised and semi-supervised objectives on large unlabeled sonar and audio datasets using masked autoencoding, JEPA, and contrastive learning.

Personal Projects

Fast Code Pruner ⚡

Task-aware context pruning for coding agents, built on a 17-layer Qwen2.5-Coder-0.5B backbone with a native vLLM serving path.

The pruner uses the normalized final-layer representation, removes three unnecessary attention branches, and merges rank-8 LoRA updates into dense weights during export.

Comparison of the original Code-Pruner and Fast Code Pruner architectures

Architecture highlights

Qwen2.5-Coder layers 1–17 with a normalized 896-dimensional final representation.
Gated PolyNorm expands 896 → 2432 before one bidirectional fusion-attention block.
CRF emissions reduce 2432 → 128 → 2 for line-level keep/prune decisions.

Validation quality

Fast Code Pruner validation quality on the fixed 6,119-example seed-43 split
Model	Accuracy ↑	Precision ↑	Recall ↑	F1 ↑
fast-code-pruner	85.94%	81.49%	83.49%	82.48%
code-pruner	84.07%	80.02%	80.91%	80.46%

Serving performance

Fast Code Pruner serving performance
Model	Backend	Concurrency 1 ↑	Concurrency 16 ↑
fast-code-pruner	vLLM 0.27.0	85.0 req/s	214.4 req/s
fast-code-pruner	Hugging Face	16.01 req/s	16.03 req/s
code-pruner	Hugging Face	9.83 req/s	10.03 req/s

ControlNet for Diffusion Transformers 🎨

Built a ControlNet-like module for fine-grained text-to-image control, extending ControlNet-XS.
Outperformed Sana’s ControlNet baseline across all metrics.
Injected conditioning with zero-conv layers to preserve pretrained features.
Engineered efficient training with lazy loading & reduced memory footprint.

ControlNet evaluation results
Model	FID (↓)	LPIPS (↓)	SSIM (↑)	CLIP ↑	CLIP Aesthetic ↑
ControlNet	38.78	0.37	0.33	0.341	11.42
ControlNet-XS	34.38	0.30	0.43	0.344	12.39

Visual Question Answering 🔍

Developed a VQA pipeline inspired by LLaVA: vision encoder ➜ connector ➜ language model.
Staged training: connector first, then LoRA-fine-tuned the LLM.
Bench-tested SigLIP, MobileCLIP, DINOv2, EfficientSAM for robust visual features.
Added dynamic high-res processing via LLaVA-NeXT + s² wrapper.
Compared Gemma, Qwen, SmolLM, OpenELM for answer quality.

World Model à la Google Genie 🧞

Frame Tokenizer ➜ Latent Action Model ➜ Dynamics Model pipeline.
EfficientVit + MobileStyleGAN compression for ultra-fast tokenization / decoding.
Replaced Genie’s ST-Transformer with a quantized lightweight MLP.

Mobile Face Transformation App 📱

🏆 First place at Samsung Next MobileXGenAI Hackathon — real-time 30 fps face transformations on mobile (CoreML optimised).
Custom encoders inject facial features at multiple StyleGAN decoder layers for detailed, natural edits.
Combined pixel, perceptual, and adversarial losses for robust and identity-preserving results.
Efficient pipeline (MobileStyleGAN + EfficientFormer + CLIP) enables high-quality transformations fully on-device.
Used both w-latents and F-latents for flexible and realistic facial attribute manipulation.
App is fully edge-compatible: minimal memory footprint, no server-side inference needed.