{"componentChunkName":"component---src-templates-blog-post-tsx","path":"/r-code-and-reproducible-model-development-with-dvc","result":{"data":{"markdownRemark":{"id":"e52e4195-4241-514b-8e1e-fff9130a4038","excerpt":"

DVC or Data Version Control tool — its idea is to track\nfiles/data dependencies during model development in order to facilitate…

","html":"

DVC or Data Version Control tool — its idea is to track\nfiles/data dependencies during model development in order to facilitate\nreproducibility and track data files versioning. Most of the\nDVC tutorials provide good examples of using\nDVC with Python language. However, I realized that DVC is a\nlanguage agnostic tool and\ncan be used with any programming language. In this blog post, we will see how to\nuse DVC in R projects.

R coding — keep it simple and readable

Each development is always a combination of following steps presented below:

\n \n \n \n \n \n $\"Model$ \n \n \n \nModel development process

Because of the specificity of the process — iterative development, it is very\nimportant to improve some coding and organizational skills. For example, instead\nof having one big R file with code it is better to split code in several logical\nfiles — each responsible for one small piece of work. It is smart to track\nhistory development with\ngit\ntool. Writing “reusable code” is nice skill to have. Put comments in a code\ncan make our life easier.

Beside git, next step in further improvements is to try out and work with DVC.\nEvery time when a change/commit in some of the codes and data sets is made, DVC\nwill reproduce new results with just one bash command on a linux (or Win\nenvironment). It memorizes dependencies among files and codes so it can easily\nrepeat all necessary steps/codes instead of us worrying about the order.

R example — data and code clarification

We’ll take an Python example from\nDVC tutorial (written by Dmitry Petrov)\nand rewrite that code in R. With an example we’ll show how can DVC help during\ndevelopment and what are its possibilities.

Firstly, let’s initialize git and dvc on mentioned example and run our codes for\nthe first time. After that we will simulate some changes in the codes and see\nhow DVC works on reproducibility.

R codes can be downloaded from the\nGithub repository. A brief explanation of\nthe codes is presented below:

parsingxml.R — it takes xml that we downloaded from the web and creates\nappropriate csv file.

\n \n\n

	#!/usr/bin/Rscript
	library(XML)
	\n
	args = commandArgs(trailingOnly=TRUE)
	if (!length(args)==2) {
	stop(\"Two arguments must be supplied (input file name ,output file name - csv ext).n\", call.=FALSE)
	}
	\n
	\n
	#read XML line by line
	con <- file(args[1], \"r\")
	lines <- readLines(con, -1)
	test <- lapply(lines,function(x){return(xmlTreeParse(x,useInternalNodes = TRUE))})
	\n
	#parsing XML to get variables
	ID <- as.numeric(sapply(test,function(x){return(xpathSApply(x, \"//row\",xmlGetAttr, \"Id\"))}))
	Tags <- sapply(test,function(x){return(xpathSApply(x, \"//row\",xmlGetAttr, \"Tags\"))})
	Title <- as.character(sapply(test,function(x){return(xpathSApply(x, \"//row\",xmlGetAttr, \"Title\"))}))
	Body <- as.character(sapply(test,function(x){return(xpathSApply(x, \"//row\",xmlGetAttr, \"Body\"))}))
	text = paste(Title,Body)
	\n
	label = as.numeric(sapply(Tags,function(x){return(grep(\"python\",x))}))
	label[is.na(label)]=0
	\n
	#final data frame for export
	df <- as.data.frame(cbind(ID,label,text),stringsAsFactors = FALSE)
	df$ID=as.numeric(df$ID)
	df$label=as.numeric(df$label)
	#write to csv
	write.csv(df, file=args[2],row.names=FALSE)
	print(\"output file created....\")

\n\n\n

\n\n

\n view raw\n parsingxml.R\n hosted with ❤ by GitHub\n

traintestspliting.R — stratified sampling by target variable (here we are\ncreating test and train data set)

\n \n\n

	#!/usr/bin/Rscript
	library(caret)
	\n
	args = commandArgs(trailingOnly=TRUE)
	\n
	if (!length(args)==5) {
	stop(\"Five arguments must be supplied (input file name, splitting ratio related to test data set, seed, train output file name, test output file name).n\", call.=FALSE)
	}
	\n
	\n
	set.seed(as.numeric(args[3]))
	\n
	df <- read.csv(args[1],stringsAsFactors = FALSE)
	\n
	test.index <- createDataPartition(df$label, p = as.numeric(args[2]), list = FALSE)
	\n
	\n
	train <- df[-test.index,]
	test <- df[test.index,]
	\n
	\n
	write.csv(train, file=args[4],row.names=FALSE)
	write.csv(test, file=args[5],row.names=FALSE)
	print(\"train/test files created....\")

\n\n\n

\n\n

\n view raw\n train_test_splitting.R\n hosted with ❤ by GitHub\n

featurization.R — text mining and tf-idf matrix creation. In this part we\nare creating predictive variables.

\n \n\n

\n \n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n

	#!/usr/bin/Rscript
	library(text2vec)
	library(MASS)
	library(Matrix)
	\n
	args = commandArgs(trailingOnly=TRUE)
	\n
	if (!length(args)==4) {
	stop(\"Four arguments must be supplied ( train file (csv format) ,test data set (csv format), train output file name and test output file name - txt files ).n\", call.=FALSE)
	}
	\n
	#read input files
	df_train = read.csv(args[1],stringsAsFactors = FALSE)
	df_test = read.csv(args[2],stringsAsFactors = FALSE)
	\n
	#create vocabulary - words
	prep_fun = tolower
	tok_fun = word_tokenizer
	\n
	it_train = itoken(df_train$text, preprocessor = prep_fun, tokenizer = tok_fun, ids = df_train$ID, progressbar = FALSE)
	vocab = create_vocabulary(it_train,stopwords = stop_words)
	\n
	#clean vocabualary - use only 5000 terms
	pruned_vocab <- prune_vocabulary(vocab, max_number_of_terms=5000)
	\n
	vectorizer = vocab_vectorizer(pruned_vocab)
	dtm_train = create_dtm(it_train, vectorizer)
	\n
	#create tf-idf for train data set
	tfidf = TfIdf$new()
	dtm_train_tfidf = fit_transform(dtm_train, tfidf)
	\n
	#create test tf-idf - use vocabulary that is build on train
	it_test = itoken(df_test$text, preprocessor = prep_fun, tokenizer = tok_fun, ids = df_test$ID, progressbar = FALSE)
	dtm_test_tfidf = create_dtm(it_test, vectorizer) %>%
	transform(tfidf)
	#add Id as additional column in matrices
	dtm_train_tfidf<- Matrix(cbind(label=df_train$label,dtm_train_tfidf),sparse = TRUE)
	dtm_test_tfidf<- Matrix(cbind(label=df_test$label,dtm_test_tfidf),sparse = TRUE)
	\n
	# write output - tf-idf matrices
	writeMM(dtm_train_tfidf,args[3])
	writeMM(dtm_test_tfidf,args[4])
	\n
	print(\"Two matrices were created - one for train and one for test data set\")

\n\n\n

\n\n

\n view raw\n featurization.R\n hosted with ❤ by GitHub\n

train_model.R — with created variables we are building logistic regression\n(LASSO).

\n \n\n

	#!/usr/bin/Rscript
	library(Matrix)
	library(glmnet)
	\n
	# three arguments needs to be provided - train file (.txt, matrix), seed and output name for RData file
	\n
	args = commandArgs(trailingOnly=TRUE)
	\n
	if (!length(args)==3) {
	stop(\"Three arguments must be supplied ( train file (.txt, matrix), seed and argument for RData model name).n\", call.=FALSE)
	}
	\n
	#read train data set
	trainMM = readMM(args[1])
	set.seed(as.numeric(args[2]))
	\n
	#use regular matrix, not sparse
	trainMM_reg <- as.matrix(trainMM)
	\n
	t1 = Sys.time()
	print(\"Started to train the model... \")
	glmnet_classifier = cv.glmnet(x = trainMM_reg[,2:500], y = trainMM_reg[,1],
	family = 'binomial',
	# L1 penalty
	alpha = 1,
	# interested in the area under ROC curve
	type.measure = \"auc\",
	# 5-fold cross-validation
	nfolds = 5,
	# high value is less accurate, but has faster training
	thresh = 1e-3,
	# again lower number of iterations for faster training
	maxit = 1e3)
	print(\"Model generated...\")
	print(difftime(Sys.time(), t1, units = 'sec'))
	\n
	preds = predict(glmnet_classifier, trainMM_reg[,2:500], type = 'response')[,1]
	\n
	print(\"AUC for the train... \")
	glmnet:::auc(trainMM_reg[,1], preds)
	\n
	save(glmnet_classifier,file=args[3])

\n\n\n

\n\n

\n view raw\n train_model.R\n hosted with ❤ by GitHub\n

evaluate.R — with trained model we are predicting target on test data set.\nAUC is final output which is used as evaluation metric.

\n \n\n

	#!/usr/bin/Rscript
	library(Matrix)
	library(glmnet)
	\n
	args = commandArgs(trailingOnly=TRUE)
	\n
	if (!length(args)==3) {
	stop(\"Three arguments must be supplied ( file name where model is stored (RDataname), test file (.txt, matrix) and file name for AUC output).n\", call.=FALSE)
	}
	\n
	#read test data set and model
	load(args[1])
	testMM = readMM(args[2])
	testMM_reg <- as.matrix(testMM)
	\n
	#predict test data
	preds = predict(glmnet_classifier, testMM_reg[,2:500] , type = 'response')[, 1]
	glmnet:::auc(testMM_reg[,1], preds)
	\n
	#write AUC into txt file
	write.table(file=args[3],paste('AUC for the test file is : ',glmnet:::auc(testMM_reg[,1], preds)),row.names = FALSE,col.names = FALSE)
	\n
	\n

\n\n\n

\n\n

\n view raw\n evaluate.r\n hosted with ❤ by GitHub\n

Firstly, codes from above we will download into the new folder (clone the\nrepository):

$ mkdir R_DVC_GITHUB_CODE\n$ cd R_DVC_GITHUB_CODE\n\n$ git clone https://github.com/Zoldin/R_AND_DVC

DVC installation and initialization

On the first site it seemed that DVC will not be compatible to work with R\nbecause of the fact that DVC is written in Python and as that needs/requires\nPython packages and pip package manager. Nevertheless, the tool can be used with\nany programming language, it is language agnostic and as such is excellent for\nworking with R.

Dvc installation:

$ pip3 install dvc\n$ dvc init

With code below 5 R scripts with dvc run are executed. Each script is started\nwith some arguments — input and output file names and other parameters (seed,\nsplitting ratio etc). It is important to use dvc run — with this command R\nscript are entering pipeline (DAG graph).

$ dvc import https://s3-us-west-2.amazonaws.com/dvc-share/so/25K/Posts.xml.tgz \\\n             data/\n\n# Extract XML from the archive.\n$ dvc run tar zxf data/Posts.xml.tgz -C data/\n\n# Prepare data.\n$ dvc run Rscript code/parsingxml.R \\\n                  data/Posts.xml \\\n                  data/Posts.csv\n\n# Split training and testing dataset. Two output files.\n# 0.33 is the test dataset splitting ratio.\n# 20170426 is a seed for randomization.\n$ dvc run Rscript code/train_test_spliting.R \\\n                  data/Posts.csv 0.33 20170426 \\\n                  data/train_post.csv \\\n                  data/test_post.csv\n\n# Extract features from text data.\n# Two TSV inputs and two pickle matrixes outputs.\n$ dvc run Rscript code/featurization.R \\\n                  data/train_post.csv \\\n                  data/test_post.csv \\\n                  data/matrix_train.txt \\\n                  data/matrix_test.txt\n\n# Train ML model out of the training dataset.\n# 20170426 is another seed value.\n$ dvc run Rscript code/train_model.R \\\n                  data/matrix_train.txt 20170426 \\\n                  data/glmnet.Rdata\n\n# Evaluate the model by the testing dataset.\n$ dvc run Rscript code/evaluate.R \\\n                  data/glmnet.Rdata \\\n                  data/matrix_test.txt \\\n                  data/evaluation.txt\n\n# The result.\n$ cat data/evaluation.txt

Dependency flow graph on R example

Dependency graph is shown on picture below:

\n \n \n \n \n \n $\"Dependency$ \n \n \n Dependency\ngraph

DVC memorizes this dependencies and helps us in each moment to reproduce\nresults.

For example, lets say that we are changing our training model — using ridge\npenalty instead of lasso penalty (changing alpha parameter to 0). In that case\nwill change/modify train_model.R job and if we want to repeat model\ndevelopment with this algorithm we don’t need to repeat all steps from above,\nonly steps marked red on a picture below:

\n \n \n \n \n \n $\"marked$ \n \n \n

DVC knows based on DAG graph that changed train_model.R file will only change\nfollowing files: Glmnet.RData and Evaluation.txt. If we want to see our new\nresults we need to execute only train_model.R and evaluate.R job. It is cool\nthat we don’t have to think all the time what we need to repeat (which steps).\ndvc repro command will do that instead of us. Here is a code example :

$ vi train_model.R\n$ git commit -am \"Ridge penalty instead of lasso\"\n$ dvc repro data/evaluation.txt\n\nReproducing run command for data item data/glmnet.Rdata. Args: Rscript code/train_model.R data/matrix_train.txt 20170426 data/glmnet.Rdata\nReproducing run command for data item data/evaluation.txt. Args: Rscript code/evaluate.R data/glmnet.Rdata data/matrix_test.txt data/evaluation.txt\n\n$ cat data/evaluation.txt\n\"AUC for the test file is :  0.947697381983095\"

dvc repro always re executes steps which are affected with the latest\ndeveloper changes. It knows what needs to be reproduced.

DVC can also work in an “multi-user environment” . Pipelines (dependency\ngraphs) are visible to others colleagues if we are working in a team and using\ngit as our version control tool. Data files can be shared if we set up a cloud\nand with dvc sync we specify which data can be shared and used for other\nusers. In that case other users can see the shared data and reproduce results\nwith those data and their code changes.

Summary

DVC tool improves and accelerates iterative development and helps to keep track\nof ML processes and file dependencies in the simple form. On the R example we\nsaw how DVC memorizes dependency graph and based on that graph re executes only\njobs that are related to the latest changes. It can also work in multi-user\nenvironment where dependency graphs, codes and data can be shared among multiple\nusers. Because it is language agnostic, DVC allows us to work with multiple\nprogramming languages within a single data science project.

","timeToRead":15,"fields":{"slug":"/r-code-and-reproducible-model-development-with-dvc"},"frontmatter":{"title":"R code and reproducible model development with DVC","date":"July 24, 2017","description":"In this document we will briefly explore possibilities of a new open source\ntool that could help with achieving code simplicity, readability and faster\nmodel development.\n","descriptionLong":"There are a lot of example on how to use Data Version Control (DVC) with a\nPython project. In this document I would like to see how it can be used with a\nproject in R.\n","tags":["RStats","R","DVC"],"commentsUrl":"https://discuss.dvc.org/t/r-code-and-reproducible-model-development-with-dvc/298","author":{"childMarkdownRemark":{"frontmatter":{"name":"Marija Ilić","avatar":{"childImageSharp":{"fixed":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAIAAAAC64paAAAACXBIWXMAAAsSAAALEgHS3X78AAAEkElEQVQ4yy3Q+VPSeRzHcXZnZzs0w8IjQYNUEhRF5VYQEAUEb0XFXEtyK7LD1U6PstTWXPPKQctUvgffk+/3C4jpmLaNre1O287sVLM/9M/sl21nnvP57fF+fefLYej1SPglG8OsU2yhDSYUfb64dG/g9tPpaRhESJwKMVGCCJGxaByjMDSIBggExjkoQhI4zQZC2OoauLS41NXRUSDOykw5np8p1BUVOcrNY0ODCBgA12DAH1hbhVZXwJUXwMqyn8NeCpIhFiOsB+G2xrrkw9+nHzkkST4q5sULEg5mcA/li/gjt/oxGPEDCASi7Ak/e2gN5mAETZAMQTA4Ti/6FjWy01pRSnOxuFMnaSgU2XMFFVKhnM8rleYszi/AcBD0oxCAQn4EXAtw8P8w+5IhxnuhWy8WXKkoGnQUPWzQjLpKx5t0DxpKrlQoNIIkT1sbFqAgPwoDGPuCawgHwykWk8EQAIFGrcIpP3nHUTjRpF7wVPku1fu67LNu46TbdEYpYceXZp8hEAkB2FfPLlPsLPuHJ8bGcwS8OoVwsFY15dYv9zSu9nUt97jmz9nnOp19ZpWcnzTcd4dCmQCAxfLjMcyOhyMb173eDO6B+iLRnSr1zy2mF94WuK975dqZha66uWZHb5lSwou/2N5BozQC4hgbQMU+m10OhaPtjXWC+G/blJKrZtWPZlWvreS5p3nSXdNTaRh2Vl4tVUq4h886q0mYxGACZ4NimGYxRa83VFVmxH/XbVK4i7Jq5DkdOvlDh/6GWeeQ5Xo0xf2mUtUJbndtLQEFiQBNwDQRCH3FDEWtn3W1SHiJ16zaQZtqqMZ006IdNhTer9DesOhHK00jVZVGwbHrrW00thFEIhS6TiPrMcxGkqGBvn7hce45tWym2rzQartrko8aVU9s+vma8uWm6vHaujJB0oi3J0q/YvBomNwMYy85KEaxsX52Zp6feswmTp+0mx5Xlw2aiydtloVq67zTDJ1p7TWbSkRpvvHHL5ndCLkZpbY3g9v/4wBKrSxDytzMYn7iaHXJA6vqlqZgwlA2bTXN1ZcDHnetJLuyMC8CEFvM663Q7nb49U741xhG0CBGME9nZnTSdDEvrkuTNWJVDGgL76vVTyzGFbfzXo1Zyk3otJS+IiO7kb030b29zd/ebu5zAmgQRggMZ/oun5fz4zJ58caspEsl0keO0plGy7MW+y/1FpMoNT+F222QUz7fu53377Z+f7/z5/vdDxwQIhCUXpietioluamHRYkHzDknvOUFvWV5Y3bdmK2kQ56dz0swZqa68oVD7c37oa2P+58/7n369PYzBwDxAIR21lcpTibK+Amnjh00ZiXfcmpv2JUDTu1dm7JZLixMPWIVn7BlJrcUZE+dv7hPbP6z9/eXv75w/H5kdGhYdyo5P+WgmHdAEPeNLO3QRVPeSINhxGV82KT/QZWtSztiPXXckHbUlZvuUcsnr/xE+YBPbz5wnk7N9V+6YFVIDbkitThDJuSrT6d7q3QTZ2unPI3TnqbrDn27VtauyXMppb12w2Vr2ezN2+Qi/MfG/r98UBJBtoZjogAAAABJRU5ErkJggg==","width":40,"height":40,"src":"/static/9add844328fd47c78f5df2bdfe1c56a3/4d3a4/marija_ilic.png","srcSet":"/static/9add844328fd47c78f5df2bdfe1c56a3/4d3a4/marija_ilic.png 1x,\n/static/9add844328fd47c78f5df2bdfe1c56a3/4c8bc/marija_ilic.png 1.5x,\n/static/9add844328fd47c78f5df2bdfe1c56a3/c0e17/marija_ilic.png 2x","srcWebp":"/static/9add844328fd47c78f5df2bdfe1c56a3/e145b/marija_ilic.webp","srcSetWebp":"/static/9add844328fd47c78f5df2bdfe1c56a3/e145b/marija_ilic.webp 1x,\n/static/9add844328fd47c78f5df2bdfe1c56a3/0d42c/marija_ilic.webp 1.5x,\n/static/9add844328fd47c78f5df2bdfe1c56a3/f46db/marija_ilic.webp 2x"}}}}}},"picture":{"childImageSharp":{"fluid":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsTAAALEwEAmpwYAAAC00lEQVQ4y2NgwAFcrd3BtASvAMPUOTvAbE0NYzDt5ZfBQBIIm9MKpoMn1DPE7pgEF895NoUhbGo9mB25oJXBpl+PsGG504+D6ZTpaxm0GUrBbFVJbU5laVV+mJqUmSvBdOGs00D1J7AbZGpmBmfH1LZAvLuGgdF7Vhq3g6+1uIu/g5RXezhPwKoUNpBcfGcLInicndFcFRkFptXV1Bj+//8PZmfOucyUO+0EX/Lq1eLebWVKzqXp6skblkhmzt4llDvlHDtIzfI//xlcvQMhQZCWCTEsZNYWMO3fMZ8hc+s5Bu/UBiZgdDDlTd6nnT35kFpq5Xzt2KJpBlHZPRZJJTOMUto2KOdMO2qcWVDFBXbpwm5gmHeAzYiY1wYxNHLbXbhrxcVlWHiYGbjLl9yWypp0RDmpcZlWYsNSncT6RXqJtfMN0nv3K+bPvqJuZGYmhOzLuG1zUF0YduU/Y0jHDGE//3AZdws7Sd+UQomEBQcEG/b9Z67c8YulZMd3lr6b/5kSlq4Q8EtLlHI2c5by9vGXiJhUKRi5fioT3IURsyEG9h7/zxA6c52AV/sUCcfaDqng6cvFo5fu5kCPvMAlTWw+U3LEfDvjpb1648Qj59UK9d2FhHvMwiaIoowtzxgkdRnBbP6YVAaf9DSuaSBvSDEwp0fYswTZSLEYaSvy1CXKM6WqCDEXMDMwJRfZoViWd2AzNPygLgSB8ElrwabmTjzKmj3lsGbBjOOKBVMPqcRXzDQNyWyxS29ZrZs//ahS/vQjKnnTTiqD1EbNr2E4vQPiwvjFkETP4D9rJYNGQChD9vSTYH76lKOc2dOOm+XPPquePWm/XmL1XOfUuoUOSXWL7fJmntLMm3FSK2/GCWO462YcZwhYGIIaNsFBQRDvTz4KpssWXGLMmXKYuWLTFyZkdfnTjzPnzTjFfP8xxFU5uHIKMsiacoyoPA8zzCRvGVwMAD/A8rORmcCjAAAAAElFTkSuQmCC","aspectRatio":1.4166666666666667,"src":"/static/18df1a636cbcc0b9b44625158d4de277/286b3/post-image.png","srcSet":"/static/18df1a636cbcc0b9b44625158d4de277/1f44b/post-image.png 213w,\n/static/18df1a636cbcc0b9b44625158d4de277/3e433/post-image.png 425w,\n/static/18df1a636cbcc0b9b44625158d4de277/286b3/post-image.png 850w,\n/static/18df1a636cbcc0b9b44625158d4de277/9a739/post-image.png 1275w,\n/static/18df1a636cbcc0b9b44625158d4de277/c47cc/post-image.png 1700w","srcWebp":"/static/18df1a636cbcc0b9b44625158d4de277/5c1d9/post-image.webp","srcSetWebp":"/static/18df1a636cbcc0b9b44625158d4de277/99b2d/post-image.webp 213w,\n/static/18df1a636cbcc0b9b44625158d4de277/23220/post-image.webp 425w,\n/static/18df1a636cbcc0b9b44625158d4de277/5c1d9/post-image.webp 850w,\n/static/18df1a636cbcc0b9b44625158d4de277/5e720/post-image.webp 1275w,\n/static/18df1a636cbcc0b9b44625158d4de277/35cfd/post-image.webp 1700w","sizes":"(max-width: 850px) 100vw, 850px","presentationWidth":850}}},"pictureComment":"DAG on R example"}}},"pageContext":{"next":{"fields":{"slug":"/data-version-control-in-analytics-devops-paradigm"},"frontmatter":{"title":"Data Version Control in Analytics DevOps Paradigm"}},"previous":{"fields":{"slug":"/how-a-data-scientist-can-improve-his-productivity"},"frontmatter":{"title":"How A Data Scientist Can Improve His Productivity"}},"currentPage":19,"slug":"/r-code-and-reproducible-model-development-with-dvc"}}}