Combined Machine Learning Algorithms and Finding the Performance

Requirement Details

Implement Adaboost using two different week classifier ( Decision Tree and Perceptron) and compare the results.

1. Describe the algorithm

2. Submit the code for the implementation.

3. Provide a table showing the performance of each classifier, as well as the combined performance of both classifiers.

Will need the algorithm of each classifier :

• decision tree + adaboost

• pla +adaboost

Implementation

AdaBoost

AdaBoost, short for “Adaptive Boosting,” is a boosting ensemble machine learning algorithm, and was one of the first successful boosting approaches. In this, I implement Adaboost with Decision Tree and Perceptron. This is a Ensemble Algorithm which used for classification and regression problems.

Real the columns

col_names=["Class Name",
"handicapped-infants",
"water-project-cost-sharing",
"adoption-of-the-budget-resolution",
"physician-fee-freeze",
"el-salvador-aid",
"religious-groups-in-schools",
"anti-satellite-test-ban",
"aid-to-nicaraguan-contras",
"mx-missile",
"immigration",
"synfuels-corporation-cutback",
"education-spending",
"superfund-right-to-sue",
"crime",
"duty-free-exports",
"export-administration-act-south-africa"]

Read Data

import pandas as pd
# reading csv files
data =  pd.read_csv('/content/house-votes-84(1) (1).data',names=col_names)
print(data)

Output:

Change it to the pandas DataFrame

df = pd.DataFrame(data)
df

Output:

Describe the dataset

df.describe()

Output:

Show the information of dataset

df.info()

output:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 435 entries, 0 to 434
Data columns (total 17 columns):
 #   Column                                  Non-Null Count  Dtype 
---  ------                                  --------------  ----- 
 0   Class Name                              435 non-null    object
 1   handicapped-infants                     435 non-null    object
 2   water-project-cost-sharing              435 non-null    object
 3   adoption-of-the-budget-resolution       435 non-null    object
 4   physician-fee-freeze                    435 non-null    object
 5   el-salvador-aid                         435 non-null    object
 6   religious-groups-in-schools             435 non-null    object
 7   anti-satellite-test-ban                 435 non-null    object
 8   aid-to-nicaraguan-contras               435 non-null    object
 9   mx-missile                              435 non-null    object
 10  immigration                             435 non-null    object
 11  synfuels-corporation-cutback            435 non-null    object
 12  education-spending                      435 non-null    object
 13  superfund-right-to-sue                  435 non-null    object
 14  crime                                   435 non-null    object
 15  duty-free-exports                       435 non-null    object
 16  export-administration-act-south-africa  435 non-null    object
dtypes: object(17)
memory usage: 57.9+ KB

#replace '?' using np.nan
df[df.loc[:,:]=="?" ]= np.nan
#replace 'n' with 0
df[df.loc[:,:]=="n" ]= 0
#replace 'y' with 1
df[df.loc[:,:]=="y" ]= 1

Finding sum of missing value

#missing values
df.isna().sum()

Output:

Class Name                                  0
handicapped-infants                        12
water-project-cost-sharing                 48
adoption-of-the-budget-resolution          11
physician-fee-freeze                       11
el-salvador-aid                            15
religious-groups-in-schools                11
anti-satellite-test-ban                    14
aid-to-nicaraguan-contras                  15
mx-missile                                 22
immigration                                 7
synfuels-corporation-cutback               21
education-spending                         31
superfund-right-to-sue                     25
crime                                      17
duty-free-exports                          28
export-administration-act-south-africa    104
dtype: int64

Encoding the dataset columns

from sklearn.preprocessing import OrdinalEncoder
ord_enc = OrdinalEncoder()
from sklearn.preprocessing import OrdinalEncoder
ord_enc = OrdinalEncoder()
for column in col_names:
    df[column] = ord_enc.fit_transform(df[[column]])

Import Libraries

from sklearn.ensemble import AdaBoostClassifier
from sklearn import datasets
# Import train_test_split function
from sklearn.model_selection import train_test_split
#Import scikit-learn metrics module for accuracy calculation
from sklearn import metrics

Split Dataset

X=df.iloc[:,1:]
y=df.iloc[:,1]

Split Dataset

# 70% training and 30% test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.35) 

ADB + DT

from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import Perceptron
dt= DecisionTreeClassifier()
pla = Perceptron()

# Create adaboost classifer object
Adb_dt = AdaBoostClassifier(n_estimators=50,
                         learning_rate=1,
                         base_estimator = dt
                         )
# Train Adaboost Classifer
model = Adb_dt.fit(X_train, y_train)
#Predict the response for test dataset
y_pred = model.predict(X_test)

Find the accuracy

print("Accuracy:",metrics.accuracy_score(y_test, y_pred))

Confusion Matrics & Classification Report

from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report

# confusion matrix
matrix = confusion_matrix(y_test, y_pred)
print('Confusion matrix : \n',matrix)

Confusion matrix : [[88 0] [ 0 65]]

Classification Report

matrix = classification_report(y_test, y_pred)
print('Classification report : \n',matrix)

Output:

lassification report : 
               precision    recall  f1-score   support

         0.0       1.00      1.00      1.00        88
         1.0       1.00      1.00      1.00        65

    accuracy                           1.00       153
   macro avg       1.00      1.00      1.00       153
weighted avg       1.00      1.00      1.00       153

pla +adaboost

# Create adaboost classifer object
Adb_pla = AdaBoostClassifier(base_estimator=Perceptron(), n_estimators=15, algorithm='SAMME')
# Train Adaboost Classifer
model = Adb_pla.fit(X_train, y_train)

#Predict the response for test dataset
y_pred = model.predict(X_test)

Print the accuracy

print("Accuracy:",metrics.accuracy_score(y_test, y_pred))

Confusion Matrix

# confusion matrix
matrix = confusion_matrix(y_test, y_pred)
print('Confusion matrix : \n',matrix)

Output: [[88 0] [ 0 65]]

Classification Report

matrix = classification_report(y_test, y_pred)
print('Classification report : \n',matrix)

Output:

Classification report : 
               precision    recall  f1-score   support

         0.0       1.00      1.00      1.00        88
         1.0       1.00      1.00      1.00        65

    accuracy                           1.00       153
   macro avg       1.00      1.00      1.00       153
weighted avg       1.00      1.00      1.00       153

If you need any programming assignment help in Machine Learning Assignment, Machine Learning Homework or Machine Learning Projet.

Send your request at realcode4you@gmail.com and get instant help with an affordable price.

We are always focus to delivered unique or without plagiarism code which is written by our highly educated professional which provide well structured code within your given time frame.

If you are looking other programming language help like C, C++, Java, Python, PHP, Asp.Net, NodeJs, ReactJs, etc. with the different types of databases like MySQL, MongoDB, SQL Server, Oracle, etc. then also contact us.