My Customized Design your own pixel art Pendant

thingiverse

The Mushroom data. This is the 1999 University of California-Irvine Machine Learning repository's first data set. **Importing libraries** ```python import pandas as pd from sklearn.model_selection import train_test_split ``` This dataset has one dependent variable which can take one out of two possible values: edible (class = 0) or poisonous (class=1). The following code reads in the Mushroom.csv and assigns a numerical representation to each category. In addition, all strings will be assigned an appropriate number. It converts all missing value placeholders ("?" in the text) into a special float (-0.0 for unknown string categories). ```python Mushrooms_df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/mushroom/agaricus-lepiota.data',header=None) numerical_map = {'e':-3, 'f': -1, 'p': 2} map_to_int = {} categoricals = [5] for c in range(len(Mushrooms_df.columns)) : for k in numerical_map.keys(): map_to_int[Mushrooms_df.columns[c]] = 0 if str(Mushrooms_df[Mushrooms_df.columns[c]][j]) != 'e' else numerical_map[str(k)] for i in range(len(categoricals)) : map_to_int[Mushrooms_df.columns[categoricals[i]]] += j + 2 # Convert categorical values to numerical. Mushroom_int_df = Mushrooms_df.replace({'?' : -0.0},regex=True) cat_int_list = list(map_to_int.values()) ``` We then transform the categories (cap-shape,cap-surface, cap-color,gill-color, ring-number,veil-type) and numbers (-0.0 etc) in the 2d arrays `Mushroom1` or `Mushrooms2`. ```python def one_hot(df,column,classes): # Get list of unique elements. all_cats = df[column].unique() for cat in all_cats: map_to_int[cat] = len(all_cats) + j col_int_dict = {str(i): i+0 for i,j in enumerate(classes)} # Get integer dictionary int_col_dict= pd.DataFrame({'cat':all_cats}).map(col_int_dict) # Introduce new category in df: new_df[col] and map original column with integer key value new = df[column].replace(map_to_int, regex=True).astype(float).astype(int) # Add the following two lines cat_cols=[x+str(j) for j,x in enumerate(classes)] onehot=pd.get_dummies(df,new_df,column,cat_int_dict) return int_col_dict[all_cats], onehot column_names=["cap-shape", "cap-surface", "gill-attachment","gill-spacing" , \ "gill-size", "stalk-shape","stalk-root"] \ # List of categorical column. col_cat =['class','population', 'habitat',"veil-color",\ 'ring-number', 'ring-type' , \ 'spore-print-color',"pop-cluster"] ``` Note: You should do the exact same thing for all columns with more than 3 elements In particular this includes (1)"cap-color"(,4,"spore-print-color"(6)"pore","veil-colour"(6)"cap-colour") ```python categ_list = [] all_cats_df_list=[] for i,j in enumerate(column_names): catego_map,catdf_onehot = one_hot(Mushrooms_int,'category[j]',str(Mushroom_class[j])) categoricals.append(cat_df.columns[-1]) # Save the int and one-hot array into dataframes. ``` Mushroom class column is binary with only two elements so no need for `pd.get_dummies`. This is an appropriate encoding. Now, convert to integers: replace string with unique numbers in order: ```python numerical_map = {'e':1,'p' : 2} numerical_cols=[5] for j,i in enumerate(range(len(categoricals))) : df['feature']+= i+1 Mushrooms_final_int=Mushroom_final.map(numerical_map).replace(('-0.0') ,-11) X = Mushroom_df.iloc[:,:20].values y= Mushroom_df[[-21]] ``` Finally, we create our training sets using sklearn library's train_test_split and the X,y defined in above cell ```python from sklearn.model_selection import train_test_split x_train,x_val, y_train, y_val = train_test_split(X,Y,test_size = .25) return x_train, x_val, y_train, y_val ``` So to combine all these into one method here is the final answer. Note that you need two separate class definition since each data set will return training/test/val differently based on proportion of examples of different classes ```python # Combine all transformations (one-hot & categorization and replace "e","p",with integer for other columns with numerical entries into one dataframe and call pandas' dummies) # This data needs no scaling before it's fit to models so use that. ```