1. Explore the sigmoid function (also known as the logistic function)
2. Explore logistic regression; which uses the sigmoid function

Sigmoid or Logistic function:

Logistic regression used a special function called the sigmoid. It looks like an S-shaped curve.

For a classification task, we can start by using the linear regression model to predict y given x.

$$ f_{\mathbf{w},b}(\mathbf{x}^{(i)}) = \mathbf{w} \cdot  \mathbf{x}^{(i)} + b $$

• However, we would like the predictions of our classification model to be between 0 and 1 since our output variable y is either 0 or 1.
• This can be accomplished by using a "sigmoid function" which maps all input values to values between 0 and 1.

Let's implement the sigmoid function and see this for ourselves.

Formula for Sigmoid function:

The formula for a sigmoid function is as follows -

$$ g(z) = \frac{1}{1+e^{-z}}\tag{1} $$

In the case of logistic regression, z (the input to the sigmoid function), is the output of a linear regression model.

• In the case of a single example, z is scalar.
• in the case of multiple examples, z may be a vector consisting of m values, one for each example.
• The implementation of the sigmoid function should cover both of these potential input formats. Let's implement this in Python.

NumPy has a function called exp(), which offers a convenient way to calculate the exponential e^z of all elements in the input array (z).

It also works with a single number as an input, as shown below.

# Input is an array. 
input_array = np.array([1,2,3])
exp_array = np.exp(input_array)

print("Input to exp:", input_array)
print("Output of exp:", exp_array)

# Input is a single number
input_val = 1  
exp_val = np.exp(input_val)

print("Input to exp:", input_val)
print("Output of exp:", exp_val)