Model help using Scikit-learn when using GridSearch

GridSearchCV as @Gauthier Feuillen said is used to search best parameters of an estimator for given data. Description of GridSearchCV:-

1. gcv = GridSearchCV(pipe, clf_params,cv=cv)
2. gcv.fit(features,labels)
3. clf_params will be expanded to get all possible combinations separate using ParameterGrid.
4. features will now be split into features_train and features_test using cv. Same for labels
5. Now the gridSearch estimator (pipe) will be trained using features_train and labels_inner and scored using features_test and labels_test.
6. For each possible combination of parameters in step 3, The steps 4 and 5 will be repeated for cv_iterations. The average of score across cv iterations will be calculated, which will be assigned to that parameter combination. This can be accessed using cv_results_ attribute of gridSearch.
7. For the parameters which give the best score, the internal estimator will be re initialized using those parameters and refit for the whole data supplied into it(features and labels).

Because of last step, you are getting different scores in first and second approach. Because in the first approach, all data is used for training and you are predicting for that data only. Second approach has prediction on previously unseen data.

Basically the grid search will:

• Try every combination of your parameter grid
• For each of them it will do a K-fold cross validation
• Select the best available.

So your second case is the good one. Otherwise you are actually predicting data that you trained with (which is not the case in the second option, there you only keep the best parameters from your gridsearch)