Metadata-Version: 2.1
Name: babyGPT
Version: 1.1.1
Summary: An educational module for experimenting with unsupervised learning in large language modeling
Home-page: https://engineering.purdue.edu/kak/distBabyGPT/babyGPT-1.1.1.html
Download-URL: https://engineering.purdue.edu/kak/distBabyGPT/babyGPT-1.1.1.tar.gz
Author: Avinash Kak
Author-email: kak@purdue.edu
Maintainer: Avinash Kak
Maintainer-email: kak@purdue.edu
License: Python Software Foundation License
Keywords: language modeling,unsupervised learning
Platform: All platforms
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3.10



Consult the module API page at

      https://engineering.purdue.edu/kak/distBabyGPT/babyGPT-1.1.1.html

for all information related to this module, including information related
to the latest changes to the code.  The page at the URL shown above lists
all of the module functionality you can invoke in your own code.

::

    Creating an instance of babyGPT:

            baby_gpt = babyGPT(
                                max_seq_length = max_seq_length,
                                batch_size = batch_size,
                                embedding_size = embedding_size,
                                num_basic_decoders = num_basic_decoders,
                                num_atten_heads = num_atten_heads,
                                optimizer_params = optimizer_params,
                                num_warmup_steps = num_warmup_steps,
                                masking = masking,
                                verify_text_corpus = False,
                                path_saved_model = {"decoder" : "./saved_decoder",                                                             
                                                    "embedding_generator" : "./saved_embedding_generator",                             
                                                   },
                              )

    Since babyGPT calls on TransformerFG for language modeling, you must also construct an instance of that class:
 
            xformer = baby_gpt.TransformerFG( 
                                max_seq_length = max_seq_length,
                                embedding_size = embedding_size,
                                tokenizer_json = tokenizer_json,
                                num_warmup_steps = num_warmup_steps,
                                optimizer_params = optimizer_params,
                      )

    Within the TransformerFG module, it is the MasterDecoder class that is needed for the next token prediction for the purpose of self-supervised learning:
            
            master_decoder = baby_gpt.MasterDecoderWithMasking(
                                xformer, 
                                num_basic_decoders = num_basic_decoders,
                                num_atten_heads = num_atten_heads,
                                masking = masking
                             )
            

    Finally, here is an instance of the dataloader you're going to need:

            dataloader = baby_gpt.ArticleDatasetWithBufferedContext(
                                gpt = baby_gpt,
                                tokenizer_json = tokenizer_json,
                                context_window_size = context_window_size,
                                context_buffer_size = context_buffer_size,
                                articles_dir = articles_dir,
                         )
          
