My Research
Simplicity leads to innovation. Efficiency is key.
Improving Small Language Models Trained on Synthetic Data Using Internet-Augmented Generation
This paper explores the idea of finetuning a model trained on synthetic data for internet-augmented generation (IAG) through a Cosmo 1b finetune on a clean and diverse dataset. Results show that IAG provides more accurate and informative answers, reduces hallucinations, and provides responses with up-to-date information. Moreover, IAG enables models that are trained on only synthetic data to understand specific examples and opinions that only web data can offer. This paper contributes to the development of more ethical training of language models by minimizing the need to pretrain on copyrighted or sensitive web data, paving the way for future advancements in the field.
Text-Guided Efficient Image Generation and Amplification for Generative Adversarial Networks (TEA-GAN): An Established Technology With Modern Techniques
This paper explores the idea of efficient text-to-image generation through GANs, instead of diffusion models. GANs are shown to be less computentially expensive than diffusion models, as they require a single pass to generate an image. By combining established techniques such as Word2Vec embedding, convolutional neural networks, and the GAN architecture, the final model can generate 64x64 images in a single pass guided by text descriptions, upscaled and processed by a different model. Consisting of a generator network that takes random noise and text embeddings as input, and a discriminator network that distinguishes between real and generated images, the generator learns to synthesize realistic images matching the input text. This paper contributes to the development of more efficient generation by offering a different method of text-to-image generation.