Options -data_dir data directory. Should contain the file input.txt with input data [data/tinyshakespeare] 训练语料 -min_freq min frequent of character [0] -rnn_size size of LSTM internal state [128] -num_layers number of layers in the LSTM [2] -model for now only lstm is supported. keep fixed [lstm] -learning_rate learning rate [0.002] -learning_rate_decay learning rate decay [0.97] -learning_rate_decay_after in number of epochs, when to start decaying the learning rate [10] -decay_rate decay rate for rmsprop [0.95] -dropout dropout for regularization, used after each RNN hidden layer. 0 = no dropout [0] -seq_length number of timesteps to unroll for [50] -batch_size number of sequences to train on in parallel [50] -max_epochs number of full passes through the training data [50] -grad_clip clip gradients at this value [5] -train_frac fraction of data that goes into train set [0.95] -val_frac fraction of data that goes into validation set [0.05] -init_from initialize network parameters from checkpoint at this path [] -seed torch manual random number generator seed [123] -print_every how many steps/minibatches between printing out the loss [1] -eval_val_every every how many iterations should we evaluate on validation data? [2000] -checkpoint_dir output directory where checkpoints get written [cv] -savefile filename to autosave the checkpont to. Will be inside checkpoint_dir/ [lstm] -accurate_gpu_timing set this flag to 1 to get precise timings when using GPU. Might make code bit slower but reports accurate timings. [0] -gpuid which gpu to use. -1 = use CPU [0] -opencl use OpenCL (instead of CUDA) [0] -use_ss whether use scheduled sampling during training [1] -start_ss start amount of truth data to be given to the model when using ss [1] -decay_ss ss amount decay rate of each epoch [0.005] -min_ss minimum amount of truth data to be given to the model when using ss [0.9]
th train.lua -data_dir data/tinyshakespeare/shakespeare_input.txt -gpuid -1 vocab.t7 and data.t7 do not exist. Running preprocessing... one-time setup: preprocessing input text file data/tinyshakespeare/shakespeare_input.txt/input.txt... loading text file... /home/frank/torch/install/bin/luajit: cannot open <data/tinyshakespeare/shakespeare_input.txt/input.txt> in mode r at /home/frank/torch/pkg/torch/lib/TH/THDiskFile.c:649 stack traceback: [C]: at 0x7f9c42473540 [C]: in function 'DiskFile' ./util/CharSplitLMMinibatchLoader.lua:201: in function 'text_to_tensor' ./util/CharSplitLMMinibatchLoader.lua:38: in function 'create' train.lua:118: in main chunk [C]: in function 'dofile' ...rank/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00405d70
这里出现了问题，因为本文是中国作者按照原karpathy的char-rnn
改写的，我认为或许使用karpathy作者的原版本教程可能会更加方便一些。于是使用As a sanity check，运行：
1
th train.lua -gpuid -1
这指的是使用CPU并不指定任何参数来训练example。
15:42开始训练
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
th train.lua -gpuid -1 loading data files... cutting off end of data so that the batches/sequences divide evenly reshaping tensor... data load done. Number of data batches in train: 423, val: 23, test: 0 vocab size: 65 creating an LSTM with 2 layers setting forget gate biases to 1 in LSTM layer 1 setting forget gate biases to 1 in LSTM layer 2 number of parameters in the model: 240321 cloning rnn cloning criterion 1/21150 (epoch 0.002), train_loss = 4.19803724, grad/param norm = 5.1721e-01, time/batch = 2.3129s 2/21150 (epoch 0.005), train_loss = 3.93712133, grad/param norm = 1.4679e+00, time/batch = 2.3114s 3/21150 (epoch 0.007), train_loss = 3.43764434, grad/param norm = 9.5800e-01, time/batch = 2.3022s 4/21150 (epoch 0.009), train_loss = 3.41313742, grad/param norm = 7.5143e-01, time/batch = 2.5311s 5/21150 (epoch 0.012), train_loss = 3.33707270, grad/param norm = 6.9269e-01, time/batch = 2.4913s
Options <model> model checkpoint to use for sampling -seed random number generator's seed [123] -sample 0 to use max at each timestep, 1 to sample at each timestep [1] -primetext used as a prompt to "seed" the state of the LSTM using a given sequence, before we sample. [] -length max number of characters to sample [2000] 采样字符大小，最大2000 -temperature temperature of sampling [1] -gpuid which gpu to use. -1 = use CPU [0] 和训练时设置应该保持一致 -verbose set to 0 to ONLY print the sampled text, no diagnostics [1] -stop stop sampling when detected [
giff, Some sweet amends, aasher, had therein, not he had wot on man's friends, for her own blow: for my men's ackingly knight, it cannot hear upon yield.
GLOUCESTER: How now again? i March, that's my arm determitness, The temper of the vrowal; ere from the grove Tut wilh 'goom'd Carulea. 'Mwailich, tne shy had lost in When Ied the way The bower to a late his body, grim on you: His opicious shames a booy, infairs,' From her, I tell you, ay, as we mean him Dear 'tis a giving o' thur back's empass'd, That nrost, I'll havk him cume thee for 't.
LAONTES: 'Twi'l love thy sronowing.
VALYRAN: Beord mocdoch him for thy follight snn hours, But thank yours lodkes, my good journeding, His jealousisposour thee are both abomish That noom that's easembelland. Camest, sir, more kia; one, in this highty be the un Since of a gournor on thy friendshall swow Some painon; and I, and lord, the at the kins Wise rit hable surliments. Shd, believh gone.
voisted tleace: Tock him what all you di turn up to celent To my sistinge. Frranch, good night, your child, so fatus; Aor he shall be my trueking: Come on my quarrel of the way: Methinks the letters; for this ctome-steers Tad mousd my smodered pouncy to haw up another sense tlays underttry Tut bonscuration fair all purpose, then be vesegt me: do not, yet rustle cannot, But for thy mustered a dust, let me Tncerfact me tresmer of his father: therefore by hanging, ANd Ays, my lord: you do here in coumisant.
LORD: How lond the brown! So majp me; bonch, smmily lovely blotters, When Ie my hoeaty threat and virlume these things, Make fasting garlands dfar the sack'd my servictught Not knows the crowns: one air, Aumerle, Ere wear not so nour Bidagle? What Aphark is fury Tld meens them, faireyou consides to no more Ihis wantond frown and pollitueser'd city. Can should put him more recounders to impudesnt poison on thet hour from hunt to Rame, supp to bere Flowerd and his friend is une dewn ao pirt, You know by join'd guilty, whathout we e.
ANd Ays, my lord: you do here in coumisant.
LORD: How lond the brown! So majp me; bonch, smmily lovely blotters, When Ie my hoeaty threat and virlume these things, Make fasting garlands dfar the sack'd my servictught Not knows the crowns: one air, Aumerle, Ere wear not so nour Bidagle? What Aphark is fury Tld meens them, faire
-------------------------- ge I prithee; nor in the day of all report? Nou shall be you that lances stoson quarrel: We mits most stone, 'aim, upeed his forticent Ahen I was, yet for her, but of lovels Clarencel And past her got and father there, one weep In mine wroth nightlys, smileed. Cantleye An York, A druls marriage, that though Service hated me Their duen of iflock, we are here in berieved and more than tanise them with suterept He rt in some pickle.
SePSERTER: Or I have safe all thou depustere andthe before him.
FRIAR LAURENCE: What art my government, cosbude, and hence; if Onfrawn? provest tor my duty?
CATESBY:
KARIANA: My love noe is are with t herman and his, It should she well deeauring our consent: They hang me bointed on the king, let so two Nature by my sighsing pleasing 'jabe That leaven and grue, at her Richard's blood. More ends it likipenortnive, nor each of him ic.
SLY: How dachors, Richmend, henr dack but like it? Be long, anon since your kingdom and us, And we that aver el aunter, my eee to toucurt tomends. It this her great fawn's birds,, sir! you'er head.
PAULINA: Upternalt cost of his hands for their tricks my father, Who ts it most seunt to live te and she were all. -kill O to thy son os shall not on your childrs, one next, for she did formly consixent Above, my life, and wew me worthy deeming tvenge! My mustere be exploience, aot come n leave where ahe knees in. dear, thus wild up tilt on the county, hath be one. See this sword of thee with the deepito man, For sunier ene first sears. Where's turn on to be. Unctious blunlest terrocate doves Trades Marcius aines of hlends My's learth an old--ay.
LEONTES: Marcius?
PRONVO: You would no gue.
VOLUMNIA: Oovine s fetch to tight, thou must but loods.
HASTINGS: And was with her, nor yonder to be sworn, What are you allady that I should have purpose. What men revenge is a well patient And who seth sxoleng to knowled ed to myself; And married me in the joy: So reve I made to find me speak, how he been tou.