Collecting and preparing Textual Data for Classification


lesson1

Lesson 1: Introduction to Textual Data Collection in NLP

Blast off into the world of Textual Data Collection! 🚀 You’re about to unlock the secrets of data science in NLP. Let’s decode the mysteries together!

Introduction to NumPy Arrays

NumPy 数组简介

Let’s delve into Python’s NumPy library and focus on the centerpiece of NumPy - arrays. NumPy, an acronym for ‘Numerical Python’, specializes in efficient computations on arrays. Arrays in NumPy are more efficient than typical Python data structures.
让我们深入探讨 Python 的 NumPy 库，并将重点放在 NumPy 的核心组件—— arrays 上。NumPy 是“Numerical Python”的缩写，专门用于对数组进行高效计算。NumPy 中的数组比典型的 Python 数据结构更高效。

Meet NumPy 邂逅 NumPy

The power of NumPy lies in its fast computations on large data arrays, making it crucial in data analysis. Before we start, let’s import it:
NumPy 的强大之处在于它能够对大型数据数组进行快速计算，这使其在数据分析中至关重要。在开始之前，让我们先导入它：

Python

CopyPlay

1# Import NumPy as 'np' in Python 2import numpy as np

np is a commonly used representation for numpy.
np 是 numpy 的常用表示。

Understanding NumPy Arrays

理解 NumPy 数组

NumPy arrays, like a sorted shopping list, allow for swift computations. Arrays offer quick access to elements. Let’s create a simple one-dimensional NumPy array:
像排序过的购物清单一样， NumPy arrays 允许快速计算。数组可以快速访问元素。让我们创建一个简单的一维 NumPy 数组：

Python

CopyPlay

1# Creating a one-dimensional (1D) numpy array 2array_1d = np.array([1, 2, 3, 4, 5]) 3print(array_1d) # prints: [1 2 3 4 5]

This code creates a five-element array.
这段代码创建了一个包含五个元素的数组。

Creating Multi-Dimensional Arrays

创建多维数组

We can create multi-dimensional arrays as much as we would with a multi-day shopping list. Here, each sublist [] forms a row in the final array:
我们可以像创建多天的购物清单一样创建多维数组。在这里，每个子列表 [] 构成最终数组中的一行：

Python

CopyPlay

1# Two-dimensional (2D) numpy array 2array_2d = np.array([[1, 2, 3],[4, 5, 6]]) 3print(array_2d) 4'''prints: 5[[1 2 3] 6 [4 5 6]] 7'''

Each row in the output corresponds to a sublist in the input list.
输出中的每一行对应于输入列表中的一个子列表。

We can apply the same principle to create a three-dimensional array:
我们可以应用相同的原理来创建一个三维数组：

Python

CopyPlay

1# Three-dimensional (3D) numpy array 2array_3d = np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]]) 3print(array_3d) 4'''prints: 5[[[ 1 2 3] 6 [ 4 5 6]] 7 8 [[ 7 8 9] 9 [10 11 12]]] 10'''

Arrays Properties: Size 数组属性：大小

NumPy arrays come with a series of built-in properties that give helpful information about the structure and type of data they hold. These are accessible via the size, shape, and type fields, respectively.
NumPy 数组带有一系列内置属性，可提供有关其保存的数据结构和类型的有用信息。 可以分别通过  size, 、 shape  和  type  字段访问这些属性。

Let’s start with size. This property indicates the total number of elements in the array. Elements can be numbers, strings, etc. This becomes especially useful when working with multi-dimensional arrays where manually counting elements can be tedious.
我们先来看一下 size 。这个属性表示数组中元素的总数。元素可以是数字、字符串等等。在处理多维数组时，这个属性特别有用，因为手动计算元素数量可能很繁琐。

Python

CopyPlay

1array_3d = np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]]) 2print("Size:", array_3d.size) # Size: 12

The array above is a 3D array that contains two 2D arrays. Each of the 2D arrays has two arrays, and each of those has three elements. Therefore, the total number of elements is 2 * 2 * 3 = 12.
上面的数组是一个包含两个二维数组的三维数组。每个二维数组又包含两个数组，每个数组有三个元素。因此，元素总数为 2 * 2 * 3 = 12 。

Arrays Properties: Shape 数组属性：形状

Next, we have shape, which gives us the array’s dimensions. The shape property returns a tuple where the number of items is the dimension of the array, and the value of each item is the size of each dimension.
接下来是 shape ，它返回数组的维度。shape 属性返回一个元组，元组中元素的数量表示数组的维度数，每个元素的值表示对应维度的长度。

Python

CopyPlay

1print("Shape:", array_3d.shape) # Shape: (2, 2, 3)

In the example above, the shape (2, 2, 3) is a tuple of three values, which indicates that our array is 3D and contains two arrays, each of which includes two more arrays, each of which holds three elements.
在上面的例子中，形状 (2, 2, 3) 是一个包含三个值的元组，这表明我们的数组是三维的，它包含两个数组，每个数组又包含两个数组，每个数组又包含三个元素。

Arrays Properties: Type 数组属性：类型

Lastly is dtype, which stands for the data type. This property tells us about the elements stored in the array, whether they’re integers, floats, strings, etc.
最后是 dtype ，它代表数据类型。此属性告诉我们数组中存储的元素是什么类型，例如整数、浮点数、字符串等。

Python

CopyPlay

1print("Data type:", array_3d.dtype) # Data type: int64

For our example, the data type is int64 because our array only contains integers. If it had held floating point numbers, the dtype would have reflected that.
对于我们的示例，数据类型是 int64 ，因为我们的数组只包含整数。如果它包含浮点数，则 dtype 会反映这一点。

Understanding these properties is vital for effectively working with NumPy arrays, as they provide information about the array’s structure and content.
了解这些属性对于有效地使用 NumPy 数组至关重要，因为它们提供了有关数组结构和内容的信息。

Lesson Summary 课程总结

Great job! We have learned how to create basic and multi-dimensional NumPy arrays and examined the properties of arrays. Now, let’s move on to some exercises to practice these concepts and prepare for future data analysis challenges.
太棒了！我们已经学习了如何创建基本和多维 NumPy 数组，并检查了数组的属性。现在，让我们继续做一些练习来实践这些概念，并为未来的数据分析挑战做好准备。

Explore More of the 20 Newsgroups Dataset

Great job, Star Seeker! Now, let’s challenge you a bit more. We have the grid_power array, representing the power generated by a 1D line of solar panels in a space station. Your task is to modify the code to transform grid_power from a 1D numpy array to a 2D numpy array with 2 rows. Keep up the good work and let’s conquer the cosmos of knowledge!import numpy as np # Assume we have four solar panels grid_power = np.array([500, 700, 700, 900]) # Print the array print("\nSpaceship Grid Power:", grid_power) # Display the size, shape, and data type of our array print(“Size of Spaceship Grid Power Array:”, grid_power.size) print(“Shape of Spaceship Grid Power Array:”, grid_power.shape) print(“Data type of Spaceship Grid Power Array:”, grid_power.dtype)

Uncover the End of 20 Newsgroups Dataset

Great job, Star Seeker! Now, let’s challenge you a bit more. We have the grid_power array, representing the power generated by a 1D line of solar panels in a space station.

Your task is to modify the code to transform grid_power from a 1D numpy array to a 2D numpy array with 2 rows.

Keep up the good work and let’s conquer the cosmos of knowledge!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import numpy as np

  

# Assume we have four solar panels

grid_power = np.array([500, 700, 700, 900])

  

# Print the array

print("\nSpaceship Grid Power:", grid_power)

  

# Display the size, shape, and data type of our array

print("Size of Spaceship Grid Power Array:", grid_power.size)

print("Shape of Spaceship Grid Power Array:", grid_power.shape)

print("Data type of Spaceship Grid Power Array:", grid_power.dtype)

解题

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import numpy as np

# Assume we have four solar panels
grid_power = np.array([500, 700, 700, 900])

# Reshape the array to 2D with 2 rows
grid_power_2D = grid_power.reshape(2, -1)

# Print the array
print("\nSpaceship Grid Power (2D):", grid_power_2D)

# Display the size, shape, and data type of our array
print("Size of Spaceship Grid Power Array (2D):", grid_power_2D.size)
print("Shape of Spaceship Grid Power Array (2D):", grid_power_2D.shape)
print("Data type of Spaceship Grid Power Array (2D):", grid_power_2D.dtype)

代码解释

The given code uses the NumPy library to transform a 1-dimensional (1D) numpy array, grid_power, into a 2-dimensional (2D) numpy array with 2 rows. The original grid_power array represents the power generated by a 1D line of solar panels in a space station.

Here’s a breakdown of the code:

1. The NumPy library is imported as np.
2. A 1D numpy array grid_power is created with four elements: [500, 700, 700, 900].
3. The reshape function is used to transform the 1D array into a 2D array with 2 rows. The -1 argument is used to automatically calculate the number of columns based on the size of the original array.
4. The transformed 2D array is printed using the print function.
5. The size, shape, and data type of the transformed 2D array are displayed using the size, shape, and dtype attributes of the numpy array.

The output of the code shows that the transformed 2D array is [[500, 700], [700, 900]], which means that the power generated by the first two solar panels is [500, 700] and the power generated by the last two solar panels is [700, 900]. The size of the transformed 2D array is 4, the shape is (2, 2), and the data type is int64.

给定的代码使用 NumPy 库将一个一维（1D）numpy 数组 grid_power 转换为一个具有 2 行的二维（2D）numpy 数组。原始的 grid_power 数组表示空间站中一排太阳能电池板生成的功率。

以下是代码的详细解释：

1. 导入 NumPy 库，并将其重命名为 np。
2. 创建一个具有四个元素的一维 numpy 数组 grid_power：[500, 700, 700, 900]。
3. 使用 print 函数打印原始的一维数组。
4. 使用 size、shape 和 dtype 属性分别显示原始一维数组的大小、形状和数据类型。
5. 使用 reshape 函数将一维数组转换为具有 2 行的二维数组。-1 参数用于根据原始数组的大小自动计算列数。
6. 使用 print 函数打印转换后的二维数组。
7. 使用 size、shape 和 dtype 属性分别显示转换后的二维数组的大小、形状和数据类型。

代码的输出显示，原始一维数组的大小为 4，形状为 (4,)，数据类型为 int64。将数组转换为具有 2 行的二维数组后，大小保持为 4，形状变为 (2, 2)，数据类型保持为 int64。转换后的二维数组为 [[500, 700], [700, 900]]，这意味着前两个太阳能电池板生成的功率为 [500, 700]，后两个太阳能电池板生成的功率为 [700, 900]。

Fetch Specific Categories from Dataset

Now, we need to make some modifications. In the code snippet provided, please add the necessary code to create a NumPy array that represents the power levels of spaceship engines. Feel free to select power levels that you deem appropriate. In addition, please insert the appropriate print statements to display the size, shape, and data type of this array.

May the force be with you!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import numpy as np

  

# TODO: Declare a numpy array for the power of spaceship engines. Choose any four powers you prefer.

# TODO: Print the created array.

  

# TODO: Display the size, shape, and data type of the created array.

解题

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import numpy as np

# Declare a numpy array for the power of spaceship engines. Choosing four powers: 100, 200, 150, 300
engine_power = np.array([100, 200, 150, 300])

# Print the created array
print("Spaceship Engine Power:", engine_power)

# Display the size, shape, and data type of the created array
print("Size of Spaceship Engine Power Array:", engine_power.size)
print("Shape of Spaceship Engine Power Array:", engine_power.shape)
print("Data type of Spaceship Engine Power Array:", engine_power.dtype)

解释 The provided code is written in Python and uses the NumPy library to create and manipulate arrays. Here’s a step-by-step explanation of the code and the solution:

1. The NumPy library is imported using the import numpy as np statement.
2. A NumPy array named engine_power is declared to represent the power levels of spaceship engines. The power levels are chosen as [100, 200, 150, 300].
3. The created array is printed using the print function with a descriptive message.
4. The size, shape, and data type of the created array are displayed using the size, shape, and dtype attributes of the NumPy array.

The output of the code will be:

Spaceship Engine Power: [100 200 150 300] Size of Spaceship Engine Power Array: 4 Shape of Spaceship Engine Power Array: (4,) Data type of Spaceship Engine Power Array: int64

This output indicates that the engine_power array has a size of 4, a shape of (4,), and a data type of int64. This means that the array is a one-dimensional array with 4 elements, and each element is an integer. 提供的代码创建了一个名为 engine_power 的 NumPy 数组，用于表示太空飞船引擎的功率级别。选择的功率级别为 [100, 200, 150, 300]。然后，代码打印创建的数组并显示其大小、形状和数据类型。

Fetching the Third Article from Dataset

Excellent work so far! Now, let’s venture into the cosmos of data transformation.

Suppose you are given an array of temperatures in degrees Celsius. Your mission, should you choose to accept it, is to write a custom function to convert these temperature values from Celsius to Fahrenheit and then apply this function to the existing array.

Remember, the formula to convert Celsius to Fahrenheit is F=C∗1.8+32F=C∗1.8+32.

Prepare for liftoff!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

import numpy as np

  

# Array of temperatures in Celsius for 5 consecutive days

temperatures_c = np.array([20.5, 25.3, 19.6, 22.7, 24.1])

  

# TODO: Write a function to convert temperatures from Celsius to Fahrenheit

  

# TODO: Apply the function to the temperatures_c array and transform them to Fahrenheit scale

  

print("Temperatures in Celsius: ", temperatures_c)

# TODO: Print the transformed Fahrenheit temperatures

Let’s proceed with the task of converting an array of temperatures from Celsius to Fahrenheit using a custom function. Here’s a step-by-step explanation of the process, followed by the actual implementation in Python:

1. Define the Conversion Function:

   • The conversion formula from Celsius to Fahrenheit is ( F = C \times 1.8 + 32 ).
   • We will define a function in Python to perform this calculation.

2. Apply the Function to the Array:

   • Use NumPy to handle the array of temperatures efficiently.
   • Apply the conversion function to each element in the array using vectorized operations.

Here’s the Python code to accomplish this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

import numpy as np

# Array of temperatures in Celsius for 5 consecutive days
temperatures_c = np.array([20.5, 25.3, 19.6, 22.7, 24.1])

# Function to convert temperatures from Celsius to Fahrenheit
def celsius_to_fahrenheit(celsius_temp):
    return celsius_temp * 1.8 + 32

# Apply the function to the temperatures_c array and transform them to Fahrenheit scale
temperatures_f = celsius_to_fahrenheit(temperatures_c)

# Print the original temperatures in Celsius
print("Temperatures in Celsius: ", temperatures_c)

# Print the transformed temperatures in Fahrenheit
print("Temperatures in Fahrenheit: ", temperatures_f)

Explanation of the Code:

• Importing NumPy: NumPy is imported for efficient array handling.
• Creating the Array: temperatures_c is defined as a NumPy array containing temperatures in Celsius.
• Defining the Conversion Function: The function celsius_to_fahrenheit takes a temperature in Celsius and converts it to Fahrenheit using the formula.
• Applying the Function: The function is applied to the entire array using NumPy’s vectorized operations, which allows for efficient computation.
• Printing Results: Finally, the original Celsius temperatures and the converted Fahrenheit temperatures are printed.

You can run this code in a Python environment to see the conversion results.

Exploring Text Length in Newsgroups Dataset

Great work, space explorer! You’ve traveled across galaxies of code and honed your skills in applying functions to arrays!

We’ve established a connection with a mysterious alien civilization. Interestingly, they value the squares of numbers and square roots of numbers more than the numbers itself! They provided us with a number sequence from 1 to 10 and seek the square and the square root of each number.

In this case, no need to vectorize custom function. We can use np.sqrt function for square roots and simply square the whole array with ** 2!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import numpy as np

  

# Array of numbers

numbers = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

  

# TODO: find squares of numbers

# TODO: find square roots of numbers

# TODO: print numbers, their squares and their square roots

Lesson 2: Mastering Text Cleaning for NLP: Techniques and Applications

Buckle up for today’s journey in the world of NumPy arrays! We’ll be exploring new routes in data selection. Isn’t that exciting? Ready to embark? 🚀

Lesson Overview and Plan

Hello, Data Whizz! Today’s lesson will focus on “selecting data” in NumPy arrays—our goal is to master integer and Boolean indexing and slicing. So, let’s jump in!

Understanding Indexing in 1D Arrays

In this section, we’ll discuss “Indexing”. It’s akin to item numbering in lists, and Python implements it with a zero-based index system. Let’s see this principle in action.

1
2
3
4
5
6
7
8
9

import numpy as np

# Let's form a 1D array and select, say, the 4th element
array_1d = np.array([2, 3, 5, 7, 11, 13, 17, 19, 23, 29])
fourth_element = array_1d[3]  # Selected element: 7

# To grab the last element, use a negative index
last_element = array_1d[-1]  # Selected element: 29

Essentially, indexing is a numeric system for items in an array—relatively straightforward!

Understanding Indexing in Multi-dimensional Arrays

Are you mindful of higher dimensions? NumPy arrays can range from 1D to N-dimensions. To access specific elements, we use the index pair (i,j) for a 2D array, (i,j,k) for a 3D array, and so forth.

1
2
3
4

# Form a 2D array and get the element at row 2 (indexed 1) and column 1 (indexed 0)
array_2d = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
element = array_2d[1,0]  # Selected element: 4

In this example, we selected the first element of the second row in a 2D-array—it’s pretty simple!

Understanding Boolean Indexing

Are you ready for some magic? Enter “Boolean Indexing”, which functions like a ‘Yes/No’ filter, with ‘Yes’ representing True and ‘No’ for False.

1
2
3
4
5
6

# A boolean index for even numbers
array_1d = np.array([8, 4, 7, 3, 4, 11])
even_index = array_1d % 2 == 0
even_numbers = array_1d[even_index]
print(even_numbers)  # Output: [8 4 4]

Or we can put the condition directly into [] brackets:

1
2
3
4
5

# A boolean index for even numbers
array_1d = np.array([8, 4, 7, 3, 4, 11])
even_numbers = array_1d[array_1d % 2 == 0]
print(even_numbers)  # Output: [8 4 4]

Voila! Now, we can filter data based on custom conditions.

Understanding Complex Conditions in Boolean Indexing

Now that we’ve mastered the simple ‘Yes/No’ binary filter system, let’s up the ante with “Complex Conditions in Boolean Indexing”. This method refines our filtering process further, allowing us to set more detailed restrictions.

Imagine, for instance, that we want to create an index for even numbers greater than five. We’ll merge two conditions to yield this result:

1
2
3
4
5

# A combined boolean index for even numbers > 5
array_1d = np.array([8, 4, 7, 3, 4, 11])
even_numbers_greater_than_five = array_1d[(array_1d % 2 == 0) & (array_1d > 5)]
print(even_numbers_greater_than_five)  # Output: [8]

In this query, we used the ampersand (&) to signify intersection - i.e., we’re selecting numbers that are both even AND larger than five. Note, that simple and operator won’t work here.

Similarly, we can use the pipe operator (|) to signify union - i.e., selecting numbers that are either even OR larger than five:

1
2
3
4
5

# A combined boolean index for even numbers or numbers > 5
array_1d = np.array([8, 4, 7, 3, 4, 11])
even_numbers_or_numbers_greater_than_five = array_1d[(array_1d % 2 == 0) | (array_1d > 5)]
print(even_numbers_or_numbers_greater_than_five)  # Output: [8 4 7 4 11]

Awesome, right? This additional filtering layer empowers us to be more specific and intentional about the data we select.

A Look at Slicing in NumPy Arrays

NumPy arrays could be sliced in the same manner as the regular python list. Let’s make a quick recall. The syntax is start:stop:step, where start is the first index to choose (inclusive), stop is the last index to choose (exclusive), and step defines the step of the selection. For example, if the step=1, each element will be selected, and if step=2 – every other one will be skipped. Let’s take a look at simple examples:

1
2
3
4
5

# Select elements at index 0, 1, 2
array_1d = np.array([1, 2, 3, 4, 5, 6])
first_three = array_1d[0:3]
print(first_three)  # Output: [1 2 3]

Note that slicing is inclusive on the left and exclusive on the right.

Another example with a step parameter:

1
2
3
4
5

# Select elements at odd indices 1, 3, 5, ...
array_1d = np.array([1, 2, 3, 4, 5, 6])
every_second = array_1d[1:6:2]
print(every_second)  # Output: [2 4 6]

In this case, we choose every second element, by starting with 1 and using step=2.

Lesson Summary

Congratulations! We’ve traversed the landscape of NumPy arrays, delving into indexing, Boolean indexing, and slicing. Now, we’ll dive into some hands-on exercises. After all, practice makes perfect. Let’s go forth, data explorers!

Update String and Clean Text

Are you ready to don your Data Analyst cape, Space Voyager? I have a task that will bring us down to Earth for a short while. Let’s envisage that you are a data analyst at an HR company, and your task is to filter out employees who earn more than an average wage of $30/hr.

This seems like quite a conundrum, doesn’t it? Don’t worry; we already have a solution.

Simply click Run to see how this task can be accomplished using NumPy's data selection techniques.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import numpy as np

  

# Create a NumPy array of wage per hour

wages_per_hour = np.array([14, 16, 20, 22, 25, 28, 30, 35, 38, 40, 45, 50])

  

# Select wages greater than 30 per hour

print(wages_per_hour[wages_per_hour > 30])

• 以下是对代码的逐步解释：

  1. 使用 import numpy as np 语句导入 NumPy 库。
  2. 创建一个名为 wages_per_hour 的 NumPy 数组，用于存储一组员工的每小时工资数据。
  3. 为了筛选出赚取超过 $30/hr 的员工，使用了布尔索引技术。表达式 wages_per_hour > 30 返回一个布尔数组，其中 True 对应于满足条件的索引（即工资超过 30）。
  4. 然后使用 print 函数打印筛选后的数组。

  代码的输出将是：

  [35 38 40 45 50]

  这个输出表明，工资为 $35/hr、$38/hr、$40/hr、$45/hr 和 $50/hr 的员工的工资超过了平均工资 $30/hr。

Filling in Python Functions and Regex Patterns

Well done, Space Voyager! You’ve harnessed the Power Cosmic to select adults from a group. Now, let’s make a slight alteration to the course.

Suppose you’re the coach of a team, which consists of both adults (aged 18 and above) and minors. The challenge lies in selecting only those adults who are under 30 to form the Junior Adults Team.

Trust your knowledge of python and apply it to the existing code. Are you ready? Set? Punch it to warp speed!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import numpy as np

  

# Create a NumPy array of ages

ages = np.array([15, 18, 16, 19, 36, 34, 27, 21, 23, 25])

  

# Select adults age under 30 years

adults = ages[ages > 18]

print("Adults:", adults)

以下是对代码的逐步解释：

1. 使用 import numpy as np 语句导入 NumPy 库。
2. 创建一个名为 ages 的 NumPy 数组，用于存储一组个体的年龄。
3. 为了选择成年人（年龄18岁及以上），使用了布尔索引技术。表达式 ages > 18 返回一个布尔数组，其中 True 对应于满足条件的索引（即年龄大于18）。
4. 结果数组的成年人存储在名为 adults 的变量中。
5. 为了仅选择那些年龄小于30岁的成年人，在 adults 数组上使用了另一种布尔索引技术。表达式 adults < 30 返回一个布尔数组，其中 True 对应于满足条件的索引（即年龄小于30）。
6. 然后使用 print 函数打印筛选后的年龄小于30岁的成年人数组。

代码的输出将是：

成年人：[19 36 34 27 21 23 25] 青年成年人：[19 27 21 23 25]

这个输出表明，该组中的成年人年龄为19、36、34、27、21、23和25岁。在这些成年人中，年龄小于30岁的人年龄为19、27、21、23和25岁。

Of course, I’d be happy to help with that! To modify the code to select adults under 30, you can add an additional condition to the Boolean indexing expression. Here’s a hint on how to proceed:

1. You already have the Boolean indexing expression ages > 18 to select adults.
2. To limit the age to under 30, you can add another condition using the less than operator <.
3. Use the & operator to combine the two conditions.
4. Apply the combined condition to the ages array to select adults under 30.
5. Print the filtered array to verify that it contains the correct results.

Here’s the modified code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import numpy as np

# Create a NumPy array of ages
ages = np.array([15, 18, 16, 19, 36, 34, 27, 21, 23, 25])

# Select adults under 30 years
adults_under_30 = ages[(ages > 18) & (ages < 30)]

print("Adults under 30:", adults_under_30)

This code will select adults (ages 18 and above) who are under 30 years old and print the filtered array.

Mastering Text Cleaning with Python Regex

Good try but that’s not exactly right.

Your solution currently selects adults over 18 years but does not limit the age to under 30.

Do you need a hint on how to modify it to meet the requirement? 🤔

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import numpy as np

  

# Create a NumPy array of ages

ages = np.array([15, 18, 16, 19, 36, 34, 27, 21, 23, 25])

  

# Select adults age under 30 years

adults = ages[ages > 18]

print("Adults:", adults)

Sure, let’s modify the existing code to select only those adults who are under 30 years of age. Here is the updated code:

1
2
3
4
5
6
7
8
9

import numpy as np

# Create a NumPy array of ages
ages = np.array([15, 18, 16, 19, 36, 34, 27, 21, 23, 25])

# Select adults aged 18 and above, but under 30 years
junior_adults = ages[(ages >= 18) & (ages < 30)]
print("Junior Adults:", junior_adults)

In this code, we use a boolean condition to select ages that are both 18 and above but less than 30. Running this will produce the output:

1
2

Junior Adults: [19 27 21 23 25]

This way, only the adults who are under 30 are selected for the Junior Adults Team.

Implement Text Cleaning on Dataset

Great job, Space Explorer! We’re planning to embark on a more advanced extraction mission. Imagine we are choosing a crew of space companions – cool space dogs, just like me! Within the field of weight analysis, envision a scenario where we’re tasked to identify dogs whose weight exceeds 20 kg or is under 10 kg. The information will enable us to concentrate our study on this group and assess their potential health risks.
太空探险家，干得好！我们正计划开展一项更先进的提取任务。想象一下，我们正在挑选太空伙伴——就像我一样的酷酷的太空狗！在体重分析领域，设想一个场景，我们需要识别体重超过 20 kg 或低于 10 kg 的狗。这些信息将使我们能够将研究集中在这个群体上，并评估它们的潜在健康风险。

For boolean selection with AND operand, we used the & operator. For the OR operand, use a vertical line |!
对于使用 AND 运算符的布尔选择，我们使用 & 运算符。对于 OR 运算符，请使用竖线 | ！

Are you ready to code? Let’s get started!
准备开始编程了吗？我们开始吧！

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import numpy as np

  

# Create a NumPy array

weight = np.array([12, 15, 11, 9, 23, 10, 12, 20, 25, 25, 8, 15, 4])

  

# TODO: Select weights greater than 20 or less than 10

解释 Absolutely! Let’s identify the space dogs whose weight exceeds 20 kg or is under 10 kg using the | operator for the OR condition. Here is the updated code:

1
2
3
4
5
6
7
8
9

import numpy as np

# Create a NumPy array
weight = np.array([12, 15, 11, 9, 23, 10, 12, 20, 25, 25, 8, 15, 4])

# Select weights greater than 20 or less than 10
selected_weights = weight[(weight > 20) | (weight < 10)]
print("Selected Weights:", selected_weights)

Running this code will produce the output:

1
2

Selected Weights: [ 9 23 25 25  8  4]

This code identifies and prints the weights of the space dogs that are either over 20 kg or under 10 kg, helping you to focus your study on this group for potential health risk assessments.

Mastering Text Cleaning with Python Regex on a Dataset

Bravo, fellow space explorer! We’ve unraveled the mysteries of data selection in NumPy arrays. Now, in this quadrant’s final test, let’s imagine that you are a space health officer.

You have an array of crew members’ badge numbers and need to choose some subgroup of people for a health check-up. Write a code that selects crew members with even badge number lower than 33. After this, select the first three crew members from the obtained list, using array slicing. Remember, the syntax is [start:stop:step], just like with python lists!
你有一个数组，存储着船员的徽章号码，你需要从中选择一些人进行健康检查。编写代码，选择徽章号码为偶数且小于  33  的船员。然后，使用数组切片从获得的列表中选择前三名船员。记住，语法是  [start:stop:step]  ，就像 Python 列表一样！

Let’s rev up our coding hyperdrive!
让我们加速代码引擎！

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import numpy as np

  

badges = np.array([12, 14, 15, 20, 32, 33, 40, 42])

  

# TODO: Using a condition-based selection, filter badges that has even number lower than 33

  

# TODO: Print the filtered numbers.

解释 Let’s embark on this final mission. We’ll filter the crew members with even badge numbers lower than 33 and then select the first three members from the resulting list using array slicing. Here is the code to achieve this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import numpy as np

# Array of crew members' badge numbers
badges = np.array([12, 14, 15, 20, 32, 33, 40, 42])

# Filter badges that have even numbers and are lower than 33
filtered_badges = badges[(badges % 2 == 0) & (badges < 33)]

# Select the first three crew members from the filtered list
selected_crew = filtered_badges[:3]

# Print the filtered numbers
print("Filtered Badges:", filtered_badges)
print("Selected Crew for Health Check-up:", selected_crew)

Running this code will produce the following output:

1
2
3

Filtered Badges: [12 14 20 32]
Selected Crew for Health Check-up: [12 14 20]

This code successfully filters the badge numbers to find those that are even and less than 33, and then selects the first three members from this filtered list for the health check-up.

Lesson 3: Removing Stop Words and Stemming in Text Preprocessing

Introduction and Lesson Overview

Welcome! In today’s lesson, we’ll learn how to use functions with NumPy arrays, a crucial concept for managing large amounts of data. This ability facilitates efficient data processing — a teacher calculating class averages or a business analyst summarising company sales illustrates the practical use of these skills. Are you ready? Let’s get started! 欢迎！在今天的课程中，我们将学习如何对 NumPy 数组使用函数，这对管理大量数据至关重要。这种能力有助于高效的数据处理——教师计算班级平均成绩或业务分析师汇总公司销售额，都体现了这些技能的实际应用。准备好了吗？让我们开始吧！

Arithmetic Operations with NumPy Arrays

NumPy 数组的算术运算

Two arrays of the same shape can undergo basic arithmetic operations. The operations are performed element-wise, meaning they’re applied to each pair of corresponding elements. Suppose we have two grade arrays of students from two subjects. By adding these arrays, we can calculate the total grades:
形状相同的两个数组可以进行基本的算术运算。这些运算以元素方式执行，这意味着它们应用于每对对应的元素。假设我们有两个科目学生的成绩数组。通过将这些数组相加，我们可以计算总成绩：

1
2
3
4
5
6
7

subject1_grades = np.array([88, 90, 75, 92, 85])
subject2_grades = np.array([92, 85, 78, 90, 88])

total_grades = subject1_grades + subject2_grades

print("Total grades:", total_grades)  # Output: Total grades: [180 175 153 182 173]

The two arrays are added element-wise in this code to calculate the total grades.
在这段代码中，两个数组按元素相加来计算总成绩。

Introduction to Universal Functions (ufuncs)

通用函数 (ufuncs) 简介

NumPy’s Universal Functions (also called ufuncs) perform element-wise operations on arrays, including mathematical functions like sin, cos, log, and sqrt. Let’s look at a use case:
NumPy 的通用函数（也称为 ufuncs ）对数组执行元素级操作，包括 sin 、 cos 、 log 和 sqrt 等数学函数。我们来看一个用例：

1
2
3
4
5
6

angles_degrees = np.array([0, 30, 45, 60, 90])
angles_radians = np.radians(angles_degrees)
sin_values = np.sin(angles_radians)

print("Sine of angles:", sin_values)  # Output: Sine of angles: [0. 0.5 0.70710678 0.8660254 1.]

This code applies np.sin universal function to each array element. As np.sin expects its input in radians, we first transform degrees to radians with np.radians, applying it to each array element similarly.
这段代码对每个数组元素应用 np.sin 通用函数。由于 np.sin 要求输入为弧度，我们首先使用 np.radians 将角度转换为弧度，并同样将其应用于每个数组元素。

Extending NumPy with Custom Functions

使用自定义函数扩展 NumPy 功能

NumPy allows the application of a custom function to each element of the array separately by transforming the target function with np.vectorize. Let’s create a function to check the parity of a number:
NumPy 允许通过使用 np.vectorize 转换目标函数，将自定义函数分别应用于数组的每个元素。让我们创建一个函数来检查数字的奇偶性：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

def is_even(n):
    return n % 2 == 0

vectorized_is_even = np.vectorize(is_even)

numbers = np.array([1, 2, 3, 4, 5])
results = vectorized_is_even(numbers)

print("Results:", results).  # Output: Results: [False True False True False]

The vectorized_is_even function returns an array indicating whether each value in numbers is even.
vectorized_is_even 函数返回一个数组，指示 numbers 中的每个值是否为偶数。

Lesson Summary 课程总结

Well done! You’ve learned how to apply functions to NumPy arrays, perform arithmetic operations, apply statistical functions, use ufuncs, and extend NumPy with custom functions. Up next are practice exercises for further learning. Let’s proceed!
干得好！您已经学习了如何将函数应用于 NumPy 数组、执行算术运算、应用统计函数、使用 ufuncs 以及使用自定义函数扩展 NumPy。接下来是用于进一步学习的练习。让我们继续吧！

「Pratice」Switch from LancasterStemmer to PorterStemmer

Hello there, space explorer! Are you ready to venture further into the mysterious universe of data analysis with Python and NumPy? Let’s do this!

Imagine that you’ve just hosted a space-themed quiz in your galactic neighborhood. Now, you have the raw scores of the participants. However, the participants are eager to know their percentage scores.

Luckily, all the work is done! Simply press the Run button to see how we’ve calculated the percentage scores using a function applied to a NumPy array.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import numpy as np

  

# Function to calculate percentage marks

def percentage_calc(percentage_marks):

return (percentage_marks * 100) / 75

  

# Array of raw marks

raw_marks = np.array([35, 48, 60, 72, 75])

  

# Compute percentage using our function

percentage_calc = np.vectorize(percentage_calc)

percentage_marks = percentage_calc(raw_marks)

  

print("Raw Marks: ", raw_marks)

print("Percentage Marks: ", percentage_marks)

解题

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import numpy as np

  

# Function to calculate percentage marks

def percentage_calc(percentage_marks):

  return (percentage_marks * 100) / 75

  

# Array of raw marks

raw_marks = np.array([35, 48, 60, 72, 75])

  

# Compute percentage using our function

percentage_calc = np.vectorize(percentage_calc)

percentage_marks = percentage_calc(raw_marks)

  

print("Raw Marks: ", raw_marks)

print("Percentage Marks: ", percentage_marks)

output

1
2
3

Raw Marks:  [35 48 60 72 75]
Percentage Marks:  [46.66666667 64.         80.         96.         100.        ]

This code snippet demonstrates a simple yet effective way to calculate percentage scores from raw marks using NumPy. Here’s a breakdown:

1. Importing NumPy: import numpy as np imports the NumPy library, essential for array operations.

2. Defining the percentage_calc function: This function takes a single argument, percentage_marks, representing the raw mark, and calculates the corresponding percentage score. The formula used is (percentage_marks * 100) / 75, where 75 is assumed to be the total marks for the quiz.

3. Creating the raw_marks array: raw_marks = np.array([35, 48, 60, 72, 75]) defines a NumPy array containing the raw scores of the participants.

4. Vectorizing the percentage_calc function: percentage_calc = np.vectorize(percentage_calc) converts the percentage_calc function into a vectorized function. This allows it to operate on each element of the raw_marks array individually.

5. Calculating percentage marks: percentage_marks = percentage_calc(raw_marks) applies the vectorized percentage_calc function to the raw_marks array, resulting in a new array percentage_marks containing the calculated percentage scores.

6. Printing the results: Finally, print("Raw Marks: ", raw_marks) and print("Percentage Marks: ", percentage_marks) display the original raw marks and the calculated percentage scores.

This code snippet showcases the power of NumPy for efficiently performing calculations on arrays. The vectorization technique allows for concise and efficient code, making it ideal for data analysis tasks.

「Pratice」 Removing Stop Words and Punctuation from Text

Congratulations on successfully running your first code! But, what else can you accomplish? Let’s investigate!

The provided starter code applies a 20% discount to all prices. However, what if the store policy were to change, offering a 30% discount for prices greater than 350? For other prices, discount stays the same – 20%. Could you modify our code to reflect this new discount rate in our discounted prices?

It’s time to ignite those rockets!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import numpy as np

  

# Function to calculate the discount price

def discount_price(price):

return price - (price * 0.20) # 20% discount

  

# Array of original prices

prices = np.array([100, 200, 300, 400, 500])

  

# Compute discounted prices using our function

discount_price = np.vectorize(discount_price)

discounted_prices = discount_price(prices)

  

print("Original Prices: ", prices)

print("Discounted Prices : ", discounted_prices)

解题

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

import numpy as np

  

# Function to calculate the discount price

def discount_price(price):

  if price > 350:

    return price - (price * 0.30) # 30% discount

  else:

    return price - (price * 0.20) # 20% discount

  

# Array of original prices

prices = np.array([100, 200, 300, 400, 500])

  

# Compute discounted prices using our function

discount_price = np.vectorize(discount_price)

discounted_prices = discount_price(prices)

  

print("Original Prices: ", prices)

print("Discounted Prices : ", discounted_prices)

output输出：

1
2
3

Original Prices:  [100 200 300 400 500]
Discounted Prices :  [ 80. 160. 240. 280. 350.]

Here’s how the code has been modified to implement the new discount policy:

1. Conditional Discount in discount_price Function:

   • Inside the discount_price function, an if statement is added to check if the price is greater than 350.
   • If price > 350, a 30% discount is applied: return price - (price * 0.30).
   • Otherwise (if price <= 350), the original 20% discount is applied: return price - (price * 0.20).

2. Rest of the Code Remains the Same:

   • The code for creating the prices array, vectorizing the discount_price function, and calculating discounted_prices remains unchanged.

Now, the code correctly applies a 30% discount for prices above 350 and a 20% discount for prices below or equal to 350. This demonstrates the flexibility of using conditional logic within functions to handle varying discount scenarios.

「Pratice」Stemming Words with PorterStemmer

「Pratice」Implementing Stopword Removal and Stemming Function

Excellent work so far! Now, let’s venture into the cosmos of data transformation.

Suppose you are given an array of temperatures in degrees Celsius. Your mission, should you choose to accept it, is to write a custom function to convert these temperature values from Celsius to Fahrenheit and then apply this function to the existing array.

Remember, the formula to convert Celsius to Fahrenheit is F

C ∗ 1.8 + 32 F=C∗1.8+32.

Prepare for lift off!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import numpy as np

# Array of temperatures in Celsius for 5 consecutive days
temperatures_c = np.array([20.5, 25.3, 19.6, 22.7, 24.1])

# TODO: Write a function to convert temperatures from Celsius to Fahrenheit

# TODO: Apply the function to the temperatures_c array and transform them to Fahrenheit scale

print("Temperatures in Celsius: ", temperatures_c)

# TODO: Print the transformed Fahrenheit temperatures

Let’s proceed with the task of converting an array of temperatures from Celsius to Fahrenheit using a custom function. Here’s a step-by-step explanation of the process, followed by the actual implementation in Python:

1. Define the Conversion Function:

   • The conversion formula from Celsius to Fahrenheit is ( F = C \times 1.8 + 32 ).
   • We will define a function in Python to perform this calculation.

2. Apply the Function to the Array:

   • Use NumPy to handle the array of temperatures efficiently.
   • Apply the conversion function to each element in the array using vectorized operations.

Here’s the Python code to accomplish this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

import numpy as np

# Array of temperatures in Celsius for 5 consecutive days
temperatures_c = np.array([20.5, 25.3, 19.6, 22.7, 24.1])

# Function to convert temperatures from Celsius to Fahrenheit
def celsius_to_fahrenheit(celsius_temp):
    return celsius_temp * 1.8 + 32

# Apply the function to the temperatures_c array and transform them to Fahrenheit scale
temperatures_f = celsius_to_fahrenheit(temperatures_c)

# Print the original temperatures in Celsius
print("Temperatures in Celsius: ", temperatures_c)

# Print the transformed temperatures in Fahrenheit
print("Temperatures in Fahrenheit: ", temperatures_f)

Explanation of the Code:

• Importing NumPy: NumPy is imported for efficient array handling.
• Creating the Array: temperatures_c is defined as a NumPy array containing temperatures in Celsius.
• Defining the Conversion Function: The function celsius_to_fahrenheit takes a temperature in Celsius and converts it to Fahrenheit using the formula.
• Applying the Function: The function is applied to the entire array using NumPy’s vectorized operations, which allows for efficient computation.
• Printing Results: Finally, the original Celsius temperatures and the converted Fahrenheit temperatures are printed.

You can run this code in a Python environment to see the conversion results.

「Pratice」Cleaning and Processing the First Newsgroup Article

Great work, space explorer! You’ve traveled across galaxies of code and honed your skills in applying functions to arrays!

We’ve established a connection with a mysterious alien civilization. Interestingly, they value the squares of numbers and square roots of numbers more than the numbers itself! They provided us with a number sequence from 1 to 10 and seek the square and the square root of each number.

In this case, no need to vectorize custom function. We can use np.sqrt function for square roots and simply square the whole array with ** 2!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

  

import numpy as np

  

  

# Array of numbers

  

numbers = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

  

  

# TODO: find squares of numbers

  

# TODO: find square roots of numbers

  

# TODO: print numbers, their squares and their square roots

  

Sure! Let’s proceed with finding the squares and square roots of the numbers from 1 to 10. We’ll use NumPy’s sqrt function for the square roots and the ** 2 operator for squaring the numbers. Here’s the complete Python code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import numpy as np

# Array of numbers
numbers = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

# Find squares of numbers
squares = numbers ** 2

# Find square roots of numbers
square_roots = np.sqrt(numbers)

# Print numbers, their squares, and their square roots
print("Numbers: ", numbers)
print("Squares: ", squares)
print("Square Roots: ", square_roots)

Explanation:

• Array of Numbers: The array numbers contains the sequence from 1 to 10.
• Squares: Using the ** 2 operator, we compute the square of each number in the array.
• Square Roots: Using the np.sqrt function, we compute the square root of each number in the array.
• Printing Results: Finally, the original numbers, their squares, and their square roots are printed.

You can run this code in a Python environment to see the results. Here’s what the output should look like:

1
2
3
4
5

Numbers:  [ 1  2  3  4  5  6  7  8  9 10]
Squares:  [  1   4   9  16  25  36  49  64  81 100]
Square Roots:  [1.         1.41421356 1.73205081 2.         2.23606798 2.44948974
 2.64575131 2.82842712 3.         3.16227766]

Lesson 4: Unleashing the Power of n-grams in Text Classification

Lesson Overview 课程概述

Hello, students! Today, we’re exploring the pandas DataFrame, a powerhouse structure in data analysis with Python. We’ll contrast it with NumPy arrays and teach you how to build a DataFrame. Additionally, we’ll delve into its integral parts and data types.
同学们，大家好！今天，我们将探索 Pandas DataFrame，它是 Python 数据分析中的强大结构。我们会将其与 NumPy 数组进行对比，并教您如何构建 DataFrame。此外，我们还将深入研究其组成部分和数据类型。

Introduction to the Pandas Library

Pandas 库简介

The pandas library is Python’s solution for tabular data operations, packing more punch for data analysis than NumPy, which is skewed towards numerical computations. Pandas houses two fundamental data structures: the Series (1D) and the DataFrame (2D). Often, the DataFrame is the go-to choice. Let’s start by importing pandas: Pandas 库是 Python 用于表格数据操作的解决方案，它比偏向于数值计算的 NumPy 库在数据分析方面功能更强大。Pandas 包含两种基本数据结构： Series （一维）和 DataFrame （二维）。通常， DataFrame 是首选。让我们从导入 pandas 开始：

1
2

import pandas as pd

Here, pd serves as a standard alias for pandas.
这里， pd 是 pandas 的标准别名。

Creating a DataFrame 创建数据帧

Building a DataFrame in pandas is straightforward. It can be created from a dictionary, list, or NumPy array. Here’s an example of creating a DataFrame from a dictionary:
在 pandas 中构建 DataFrame 非常简单。它可以从字典、列表或 NumPy 数组创建。以下是如何从字典创建 DataFrame 的示例：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

student_data = {  
    'Name': ['Sara', 'Ray', 'John'],
    'Age': [15, 16, 17],
}

df = pd.DataFrame(student_data)

print(df)

'''Output:
   Name  Age
0  Sara   15
1   Ray   16
2  John   17
'''

In the student_data dictionary, each (key, value) pair becomes a DataFrame column. The DataFrame automatically assigns an index (0-2) to each row, but we can also specify our own if we choose to do so.
在 student_data 字典中，每个 (键, 值) 对都会成为 DataFrame 的一列。DataFrame 会自动为每一行分配一个索引 (0-2)，但我们也可以选择指定自己的索引。

Adding a Column to the DataFrame

为数据帧添加列

Adding a new column to an existing DataFrame is straightforward: just like with dictionary, simply refer to a new key and assign an array to it. Let’s add information about student’s grades to our data:
向现有 DataFrame 添加新列很简单：就像使用字典一样，只需引用一个新键并为其分配一个数组即可。让我们将有关学生成绩的信息添加到我们的数据中：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

df['Grade'] = [9, 10, 7]

print(df)

'''Output:
   Name  Age  Grade
0  Sara   15      9
1   Ray   16     10
2  John   17      7
'''

The 'Grade' column has been successfully added to the DataFrame.
'Grade' 列已成功添加到 DataFrame 中。

Classification of DataFrame Parts

DataFrame 部分的分类

A DataFrame consists of three components: data, index, and columns. Pandas neatly organize data into rows and columns — each row is a record, and each column is a feature (such as Name, Age, or Grade). The index, much like the index of a book, aids in data location. Columns serve as labels for the features in our data. Let’s analyze these components with our student data:
DataFrame 由三个部分组成：数据、索引和列。Pandas 将数据整齐地组织成行和列——每一行都是一条记录，每一列都是一个特征（例如姓名、年龄或年级）。索引就像书的索引一样，有助于数据定位。列充当数据中特征的标签。让我们用学生数据来分析这些组成部分：

1
2
3

print(df.index)  # Output: RangeIndex(start=0, stop=3, step=1)
print(df.columns)  # Output: Index(['Name', 'Age', 'Grade'], dtype='object') 

Our dataset contains row labels (index) from 0 to 2 and column labels (columns) of ‘Name’, ‘Age’, and ‘Grade’.
我们的数据集包含从 0 到 2 的行标签（索引）以及“姓名”、“年龄”和“年级”的列标签（列）。

Accessing DataFrame Columns

访问 DataFrame 列

One of the useful features of pandas DataFrame is its flexibility in accessing individual columns of data. You can select a single column, much like how you access a dictionary item by its key. Let’s use our student_data DataFrame again:
pandas DataFrame 的一个有用特性是它在访问单个数据列方面的灵活性。您可以选择单个列，就像通过键访问字典项一样。让我们再次使用我们的 student_data DataFrame：

1
2
3
4
5
6
7
8
9

print(df['Name'])

'''Output:
0    Sara
1     Ray
2    John
Name: Name, dtype: object
'''

As you can see, we now only have the ‘Name’ column content. Notice that pandas kept the index intact to retain information about the original rows.
如您所见，我们现在只有“姓名”列的内容。请注意，pandas 保留了索引不变，以保留有关原始行的信息。

To access multiple columns at once, pass a list with column names:
要访问多个列，请传递一个包含列名的列表：

1
2
3
4
5
6
7
8
9

print(df[['Name', 'Grade']])

'''Output:
   Name  Grade
0  Sara      9
1   Ray     10
2  John     11
'''

Adding a list of columns as input, we obtain a new DataFrame that only includes the students’ ‘Name’ and ‘Grade’.
输入列名列表，我们可以得到一个新的 DataFrame，其中仅包含学生的“姓名”和“年级”。

Remember: when handling a single column, pandas will return a Series. When dealing with multiple columns, pandas will return a DataFrame. Any new DataFrame or Series changes will not affect the original data.
记住：处理单列时，pandas 将返回 Series。处理多列时，pandas 将返回 DataFrame。任何新的 DataFrame 或 Series 更改都不会影响原始数据。

Understanding Data Types 理解数据类型

The strength of a DataFrame lies in its ability to accommodate multiple data types. Using the DataFrame.dtypes property, we can ascertain the data type of each column. Let’s look at the data types of out DataFrame:
DataFrame 的优势在于它能够适应多种数据类型。使用 DataFrame.dtypes 属性，我们可以确定每一列的数据类型。让我们看一下 DataFrame 的数据类型：

1
2
3
4
5
6
7

print(df.dtypes)

Name     object
Age       int64
Grade     int64
dtype: object

Note, that strings are represented as the object data type.
注意，字符串以 object 数据类型表示。

Unlike NumPy arrays, which harbor single-type data, pandas DataFrames can handle and manipulate different data types.
与只能处理单一数据类型的 NumPy arrays 不同，pandas DataFrames 可以处理和操作多种数据类型。

Lesson Summary 课程总结

And there we have it! We’ve traversed the fascinating terrain of the pandas DataFrame. We’ve compared it with NumPy arrays, learned how to build a DataFrame, and explored its structure and diverse data types. Now, it’s time to shift gears to some hands-on practice. You’ll master the art of creating pandas’ DataFrame, solidifying your understanding and bolstering your confidence.
好了！我们已经穿越了 pandas DataFrame 的奇妙世界。我们将它与 NumPy arrays 进行了比较，学习了如何构建 DataFrame ，并探索了它的结构和多样化的数据类型。现在，让我们开始实践吧。您将掌握创建 pandas DataFrame, 的技巧，巩固您的理解并增强您的信心。

「Pratice」Generating Bigrams and Trigrams with NLP

Astronauts, let’s beam the classroom data into our trusty spaceship, the pandas DataFrame! This handy tool will help us organize and analyze the students’ test scores for different subjects. Click on the Run button to see just how easy it is!
宇航员们，让我们将课堂数据传送进我们可靠的宇宙飞船“熊猫号”里。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import pandas as pd

  

classroom_data = {

'StudentName': ['Amy', 'Bob', 'Eva', 'Jake'],

'Subject': ['Math', 'English', 'Science', 'Arts'],

'TestScore': [89, 93, 91, 97]

}

  

classroom_df = pd.DataFrame(classroom_data)

print("\nClassroom DataFrame:\n", classroom_df)

  

classroom_df['ParticipationScore'] = [9, 10, 8, 10]

print("\n Updated DataFrame with 'ParticipationScore':\n", classroom_df)

「Pratice」 Generating Bigrams and Trigrams from Text Data

Absolutely! Let’s organize and analyze the classroom data using a pandas DataFrame. Here’s the complete Python code for creating the DataFrame, adding a new column for participation scores, and displaying the updated DataFrame:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import pandas as pd

# Classroom data
classroom_data = {
    'StudentName': ['Amy', 'Bob', 'Eva', 'Jake'],
    'Subject': ['Math', 'English', 'Science', 'Arts'],
    'TestScore': [89, 93, 91, 97]
}

# Create DataFrame
classroom_df = pd.DataFrame(classroom_data)
print("\nClassroom DataFrame:\n", classroom_df)

# Add ParticipationScore column
classroom_df['ParticipationScore'] = [9, 10, 8, 10]
print("\nUpdated DataFrame with 'ParticipationScore':\n", classroom_df)

Explanation:

• Import pandas: First, we import the pandas library.
• Define Classroom Data: The classroom data is defined as a dictionary containing student names, subjects, and test scores.
• Create DataFrame: We create a DataFrame from the classroom data using pd.DataFrame.
• Print Initial DataFrame: The initial DataFrame is printed to show the classroom data.
• Add Participation Score: A new column ParticipationScore is added to the DataFrame.
• Print Updated DataFrame: The updated DataFrame is printed to show the classroom data along with the participation scores.

Running this code will produce the following output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

Classroom DataFrame:
   StudentName  Subject  TestScore
0         Amy     Math         89
1         Bob  English         93
2         Eva  Science         91
3        Jake     Arts         97

Updated DataFrame with 'ParticipationScore':
   StudentName  Subject  TestScore  ParticipationScore
0         Amy     Math         89                  9
1         Bob  English         93                 10
2         Eva  Science         91                  8
3        Jake     Arts         97                 10

This demonstrates how easy it is to organize and analyze data using pandas DataFrame.

「Pratice」Generating Bigrams and Trigrams from Two Texts

Astronauts, let’s beam the classroom data into our trusty spaceship, the pandas DataFrame! This handy tool will help us organize and analyze the students’ test scores for different subjects. Click on the Run button to see just how easy it is!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import pandas as pd

  

classroom_data = {

'StudentName': ['Amy', 'Bob', 'Eva', 'Jake'],

'Subject': ['Math', 'English', 'Science', 'Arts'],

'TestScore': [89, 93, 91, 97]

}

  

classroom_df = pd.DataFrame(classroom_data)

print("\nClassroom DataFrame:\n", classroom_df)

  

classroom_df['ParticipationScore'] = [9, 10, 8, 10]

print("\n Updated DataFrame with 'ParticipationScore':\n", classroom_df)

输出

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

Classroom DataFrame:  
   StudentName  Subject  TestScore  
0         Amy     Math         89  
1         Bob  English         93  
2         Eva  Science         91  
3        Jake     Arts         97  
  
 Updated DataFrame with 'ParticipationScore':  
   StudentName  Subject  TestScore  ParticipationScore  
0         Amy     Math         89                   9  
1         Bob  English         93                  10  
2         Eva  Science         91                   8  
3        Jake     Arts         97                  10

解释 You’re right! This code snippet is a great introduction to using pandas DataFrames for organizing and analyzing data, especially in a space exploration context. Let’s break it down step by step:

1. Importing pandas:

import pandas as pd

• This line imports the pandas library, which is a powerful tool for data manipulation and analysis in Python. We use the as pd part to give the library a shorter alias, making it easier to use later.

2. Creating the Data Dictionary:

classroom_data = { 'StudentName': ['Amy', 'Bob', 'Eva', 'Jake'], 'Subject': ['Math', 'English', 'Science', 'Arts'], 'TestScore': [89, 93, 91, 97] }

• This creates a dictionary called classroom_data to store the information about the students.
• The dictionary has three keys:
  • StudentName: A list of student names.
  • Subject: A list of subjects.
  • TestScore: A list of test scores.

3. Creating the DataFrame:

classroom_df = pd.DataFrame(classroom_data)

• This line creates a pandas DataFrame called classroom_df from the classroom_data dictionary.
• DataFrames are like tables in spreadsheets, providing a structured way to organize and work with data.

4. Printing the DataFrame:

print("\nClassroom DataFrame:\n", classroom_df)

• This line prints the classroom_df DataFrame to the console. You’ll see a neatly formatted table with the student names, subjects, and test scores.

5. Adding a New Column:

classroom_df['ParticipationScore'] = [9, 10, 8, 10]

• This line adds a new column named ParticipationScore to the classroom_df DataFrame.
• It assigns the values [9, 10, 8, 10] to the new column, representing the participation scores for each student.

6. Printing the Updated DataFrame:

print("\n Updated DataFrame with 'ParticipationScore':\n", classroom_df)

• This line prints the updated classroom_df DataFrame, now including the ParticipationScore column.

In Summary:

This code demonstrates how to:

• Import the pandas library.
• Create a dictionary to hold data.
• Convert the dictionary into a pandas DataFrame.
• Print the DataFrame.
• Add a new column to the DataFrame.

This is a basic example of how pandas can be used to work with data in a structured and efficient way. You can use pandas to perform various operations on DataFrames, such as:

• Filtering: Selecting specific rows based on conditions.
• Sorting: Arranging data in ascending or descending order.
• Aggregation: Calculating summary statistics (like mean, median, etc.).
• Joining: Combining data from multiple DataFrames.

pandas is a powerful tool for data analysis in Python, and it’s widely used in various fields, including data science, machine learning, and finance.

「Pratice」Creating Bigrams from Preprocessed Text Data

Well done, Voyager! Now, let’s tweak the code a little. Your task is to modify it to add a new column, "Experience", to the teachers_df DataFrame. For 'Experience', use the list [12, 6, 15]. Reach for the stars!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import pandas as pd

  

teachers_data = {

'Name': ['Sarah', 'Mike', 'Emma'],

'Subject': ['English', 'Math', 'Science'],

}

  

teachers_df = pd.DataFrame(teachers_data)

print("Initial DataFrame:\n", teachers_df)

  

teachers_df['Years'] = [10, 5, 8]

print("\nDataFrame after a new column:\n", teachers_df)

「Pratice」Unigrams and Bigrams from Clean 20 Newsgroups Dataset

Lesson 5: Understanding Named Entity Recognition in NLP

Adding a Row to a DataFrame

Multiple scenarios might necessitate adding new entries to our DataFrame. Let’s explore how to accomplish that:

In modern pandas, we use pd.concat() function to incorporate new rows. If you forgot to add 'Pears' to your grocery list, here’s how to do it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

new_row = pd.DataFrame({'Grocery Item': ['Pears'], 'Price per kg': [4.00]})

grocery_df = pd.concat([grocery_df, new_row]).reset_index(drop=True)

print(grocery_df)
'''Output:
  Grocery Item  Price per kg
0       Apples          3.25
1      Oranges          4.50
2      Bananas          2.75
3       Grapes          5.00
4        Pears          4.00
'''

Setting reset_index(drop=True) resets the index to default integers. Without this step, pandas will save the original dataframes’ indices, resulting in both 'Pears' and 'Apples' sharing the same index 0.

Adding Multiple Rows to a DataFrame

For multiple rows, you can concatenate them by creating a DataFrame and adding it to the original one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

new_rows = pd.DataFrame({
    'Grocery Item': ['Avocados', 'Blueberries'],
    'Price per kg': [2.5, 10.0]
})

grocery_df = pd.concat([grocery_df, new_rows]).reset_index(drop=True)

print(grocery_df)
'''Output:
  Grocery Item  Price per kg
0       Apples          3.25
1      Oranges          4.50
2      Bananas          2.75
3       Grapes          5.00
4     Avocados          2.50
5  Blueberries         10.00
'''

You may wonder why we don’t include these rows in the original dataframe. Well, it is only sometimes possible. Imagine we have two separate grocery lists coming from different sources, for instance, from separate files. In this case, the only way to combine them into one is to use pd.concat()

Removing Rows from a DataFrame

Frequently, we must delete rows from a DataFrame. To facilitate this, Pandas provides the drop() function. Suppose you want to remove 'Grapes' or both 'Apples' and 'Oranges' from your list. Here’s how:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

index_to_delete = grocery_df[grocery_df['Grocery Item'] == 'Grapes'].index

grocery_df = grocery_df.drop(index_to_delete)

print(grocery_df)
'''Output:
  Grocery Item  Price per kg
0       Apples          3.25
1      Oranges          4.50
2      Bananas          2.75
'''

Note that the .drop() method returns a new updated DataFrame instead of changing the original one. It allows you to modify the data while keeping its original state to return to it if necessary.

Removing Multiple Rows

There will be times when you will have to remove multiple rows in one go. For example, let’s say you were informed that 'Apples' and 'Oranges' are out of stock, so you need to remove them from your grocery list. The drop() function allows you to do this too.

When removing multiple rows, we utilize the .isin() function, which checks if a value exists in a particular DataFrame column. You provide it with the values you want to remove, and it outputs the indices of those rows. Let’s see it in action:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

indices_to_delete = grocery_df[grocery_df['Grocery Item'].isin(['Apples', 'Oranges'])].index

grocery_df = grocery_df.drop(indices_to_delete)

print(grocery_df)
'''Output:
  Grocery Item  Price per kg
2      Bananas          2.75
3       Grapes          5.00
'''

In this block of code, the variable indices_to_delete holds the indices of the rows where the ‘Grocery Item’ is either 'Apples' or 'Oranges'. We then pass indices_to_delete to the drop() function, which removes the corresponding rows from the DataFrame.

Keep in mind, just as with removing a single row, the drop() function here doesn’t change the original DataFrame. Instead, it returns a new DataFrame with the specified rows removed. This way, you can always revert back to the original data if needed.

Recap and Practice Announcement

Congratulations! You’ve now mastered adding and removing rows in a DataFrame, a crucial element in data manipulation. We discussed rows and their indexing and learned to add rows using pd.concat() and to remove them with drop(). Now, let’s put this into practice! The upcoming exercises will enhance your data manipulation skills, enabling you to handle more complex operations on a DataFrame. Are you ready to give them a try? Well done! Now let’s experiment a bit more with our grocery list. We’re planning to make honey toast for breakfast, so we need to add ‘Honey’ to our grocery list and remove ‘Cheese’ because it’s not needed at the moment. Examine what I’ve done, then click “Run” to execute the code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import pandas as pd

  

# Let's create a DataFrame for our grocery list

grocery_items = {'Items': ['Milk', 'Bread', 'Cheese', 'Butter'], 'Price': [2.5, 1.5, 3.5, 2.0]}

grocery_df = pd.DataFrame(grocery_items)

  

# Now let's add a new item 'Honey' to our grocery list

new_item = pd.DataFrame({'Items': ['Honey'], 'Price': [4.0]})

grocery_df = pd.concat([grocery_df, new_item]).reset_index(drop=True)

  

# But we decided we don't need 'Cheese'. Let's remove it from our list

index_to_drop = grocery_df[grocery_df['Items'] == 'Cheese'].index

grocery_df = grocery_df.drop(index_to_drop)

  

print(grocery_df)

Congratulations! You’ve now mastered adding and removing rows in a DataFrame, a crucial element in data manipulation. We discussed rows and their indexing and learned to add rows using pd.concat() and to remove them with drop(). Now, let’s put this into practice! The upcoming exercises will enhance your data manipulation skills, enabling you to handle more complex operations on a DataFrame. Are you ready to give them a try? Well done! Now let’s experiment a bit more with our grocery list. We’re planning to make honey toast for breakfast, so we need to add ‘Honey’ to our grocery list and remove ‘Cheese’ because it’s not needed at the moment. Examine what I’ve done, then click “Run” to execute the code. 做得好！现在让我们对购物清单进行更多实验。我们计划制作蜂蜜吐司作为早餐，因此我们需要将“蜂蜜”添加到我们的购物清单中并删除“奶酪”，因为目前不需要它。检查我所做的事情，然后单击“运行”来执行代码。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import pandas as pd

  

# Let's create a DataFrame for our grocery list

grocery_items = {'Items': ['Milk', 'Bread', 'Cheese', 'Butter'], 'Price': [2.5, 1.5, 3.5, 2.0]}

grocery_df = pd.DataFrame(grocery_items)

  

# Now let's add a new item 'Honey' to our grocery list

new_item = pd.DataFrame({'Items': ['Honey'], 'Price': [4.0]})

grocery_df = pd.concat([grocery_df, new_item]).reset_index(drop=True)

  

# But we decided we don't need 'Cheese'. Let's remove it from our list

index_to_drop = grocery_df[grocery_df['Items'] == 'Cheese'].index

grocery_df = grocery_df.drop(index_to_drop)

  

print(grocery_df)

输出

1
2
3
4
5
6

Items  Price  
0    Milk    2.5  
1   Bread    1.5  
3  Butter    2.0  
4   Honey    4.0

Changing the Sentence for Named Entity Recognition

Implementing Tokenization and POS Tagging

Applying Named Entity Recognition to a Sentence

Implementing a Named Entity Extraction Function

Applying NER and POS Tagging to Dataset


Feature Engineering for Text Classification

Dive deeper into the transformation of raw text data into features that machine learning models can understand. Through a practical, hands-on approach, you’ll learn everything from tokenization, generating Bag-of-Words and TF-IDF representations, to handling sparse features and applying Dimensionality Reduction techniques.

introduction and Text Data Collection

Welcome to today’s lesson! As data science and machine learning professionals, particularly in the Natural Language Processing (NLP) field, we often deal with textual data. Today, we dive into the ‘Introduction to Textual Data Collection’. Specifically, we’ll explore how to collect, understand and analyze text data using Python.

Textual data is usually unstructured, being much harder to analyze than structured data. It can take many forms, such as emails, social media posts, books, or transcripts of conversations. Understanding how to handle such data is a critical part of building effective machine learning models, especially for text classification tasks where we ‘classify’ or categorize texts. The quality of the data we use for these tasks is of utmost importance. Better, well-structured data leads to models that perform better.

The 20 Newsgroups Dataset

20 个新闻组数据集

The dataset we’ll be working with in today’s lesson is the 20 Newsgroups dataset. For some historical background, newsgroups were the precursors to modern internet forums, where people gathered to discuss specific topics. In our case, the dataset consists of approximately 20,000 documents from newsgroup discussions. These texts were originally exchanged through Usenet, a global discussion system that predates many modern Internet forums.
在今天的课程中，我们将使用的数据集是 20 个新闻组数据集。出于某种历史背景，新闻组是现代互联网论坛的前身，人们聚集在一起讨论特定主题。在我们的例子中，数据集由来自新闻组讨论的大约 20,000 个文档组成。这些文本最初是通过Usenet交换的，Usenet是一个全球讨论系统，早于许多现代互联网论坛。

The dataset is divided nearly evenly across 20 different newsgroups, each corresponding to a separate topic - this segmentation is one of the main reasons why it is especially useful for text classification tasks. The separation of data makes it excellent for training models to distinguish between different classes, or in our case, newsgroup topics.
数据集几乎平均分布在 20 个不同的新闻组中，每个新闻组对应一个单独的主题 - 这种分割是它对文本分类任务特别有用的主要原因之一。数据的分离使得训练模型能够很好地区分不同的类，或者在我们的例子中，区分新闻组主题。

From science and religion to politics and sports, the topics covered provide a diversified range of discussions. This diversity adds another layer of complexity and richness, similar to what we might experience with real-world data.
从科学和宗教到政治和体育，所涵盖的主题提供了多样化的讨论范围。这种多样性增加了另一层复杂性和丰富性，类似于我们在现实世界数据中可能遇到的情况。

Fetching and Understanding the Data Structure

获取和理解数据结构

To load this dataset, we use the fetch_20newsgroups() function from the sklearn.datasets module in Python. This function retrieves the 20 newsgroup dataset in a format that’s useful for machine learning purposes. Let’s fetch and examine the dataset.
为了加载这个数据集，我们使用 Python 中 sklearn.datasets 模块中的 fetch_20newsgroups() 函数。此函数以可用于机器学习目的的格式检索 20 个新闻组数据集。让我们获取并检查数据集。

First, let’s import the necessary libraries and fetch the data:
首先，让我们导入必要的库并获取数据：

1
2
3
4
5
6

# Importing necessary libraries
from sklearn.datasets import fetch_20newsgroups

# Fetch data
newsgroups = fetch_20newsgroups(subset='all')

「PRACTICE」

Excellent start, Space Voyager! Now, here’s a curveball. Can you fill in the missing code to create the DataFrame?

May the cosmic rays guide you!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import pandas as pd

  

# We have a list of classes and their respective teachers

classes_dict = {

'Class': ['Math', 'Science', 'English', 'Physical Education'],

'Teacher': ['Mr. James', 'Mrs. Smith', 'Ms. Williams', 'Mrs. Johnson']

}

  

# TODO: Create a DataFrame from the classes_dict

classes_df = None

  

# Display the DataFrame

print(classes_df)

解释： Excellent start, Space Voyager! Now, here’s a curveball. Can you fill in the missing code to create the DataFrame?

May the cosmic rays guide you!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import pandas as pd

  

# We have a list of classes and their respective teachers

classes_dict = {

'Class': ['Math', 'Science', 'English', 'Physical Education'],

'Teacher': ['Mr. James', 'Mrs. Smith', 'Ms. Williams', 'Mrs. Johnson']

}

  

# TODO: Create a DataFrame from the classes_dict

classes_df = None

  

# Display the DataFrame

print(classes_df)

「practice」

Time to shine, stars! Our classroom data has somewhat disappeared. Can you reconstruct it from scratch, including both the classroom data dictionary and the DataFrame? Furthermore, please add a column today, AttendancePercentage, to the DataFrame. I know you’ve got this, champ!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import pandas as pd

  

# TODO: Create a dictionary named "classroom_data" with keys: StudentName, Subject, Grade. Assign corresponding values to these keys.

  

# TODO: Create a DataFrame named "classroom_df" using the "classroom_data" dictionary. Print it.

  

# TODO: Add a new column 'AttendancePercentage' to the "classroom_df" DataFrame with values: 95, 80, 90, 85. Print the updated DataFrame.

Good try, but that’s not quite right. It seems like your code is focused on school members and teachers, but we need to modify the grocery list DataFrame instead.

Remember, we need to:

1. Add ‘Spinach’ with quantity 5.
2. Remove ‘Milk’.

Do you need help with how to start? 🤔

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

import pandas as pd

  

# create a DataFrame for a grocery list

data = {

'Item': ['Milk', 'Eggs', 'Bread', 'Butter'],

'Quantity': [2, 12, 1, 1]

}

  

grocery_df = pd.DataFrame(data)

  

# add a new row for 'Cheese'

new_row = pd.DataFrame({'Item': ['Cheese'], 'Quantity': [1]})

  

grocery_df = pd.concat([grocery_df, new_row]).reset_index(drop=True)

  

# remove 'Bread' from the list

item_to_remove = grocery_df[grocery_df['Item'] == 'Bread'].index

  

grocery_df = grocery_df.drop(item_to_remove)

  

print(grocery_df)

Sure, I can help with that. Here’s how you can modify your code to add ‘Spinach’ with quantity 5 and remove ‘Milk’:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

import pandas as pd

# Create a DataFrame for a grocery list
data = {
    'Item': ['Milk', 'Eggs', 'Bread', 'Butter'],
    'Quantity': [2, 12, 1, 1]
}

grocery_df = pd.DataFrame(data)

# Add a new row for 'Spinach'
new_row = pd.DataFrame({'Item': ['Spinach'], 'Quantity': [5]})
grocery_df = pd.concat([grocery_df, new_row]).reset_index(drop=True)

# Remove 'Milk' from the list
grocery_df = grocery_df[grocery_df['Item'] != 'Milk']

print(grocery_df)

This code first creates a new DataFrame for ‘Spinach’ and adds it to grocery_df. It then removes the row where the ‘Item’ is ‘Milk’.

「Practice」

Fantastic work, Voyager!

However, it appears we have a small bug in our shopping list code. The item “Cheese” was supposed to be removed from the list, but it remains. Can you remedy this? Remember, it’s not necessary for you to write any code from scratch; you only need to correct one or two lines of the existing code. Good luck!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

import pandas as pd

  

grocery_items = {

'Item': ['Eggs', 'Bread', 'Cheese', 'Butter'],

'Quantity': [12, 1, 1, 1]

}

  

grocery_df = pd.DataFrame(grocery_items)

  

new_items = pd.DataFrame({

'Item': ['Milk', 'Yogurt'],

'Quantity': [2, 3]

})

  

grocery_df = pd.concat([grocery_df, new_items]).reset_index(drop=True)

  

item_to_drop = grocery_df[grocery_df['Item'] == 'Cheese'].index

grocery_df.drop(item_to_drop)

  

print(grocery_df)

It looks like the drop method is not being assigned back to grocery_df. Here’s the corrected code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import pandas as pd

grocery_items = {
    'Item': ['Eggs', 'Bread', 'Cheese', 'Butter'],
    'Quantity': [12, 1, 1, 1]
}

grocery_df = pd.DataFrame(grocery_items)

new_items = pd.DataFrame({
    'Item': ['Milk', 'Yogurt'], 
    'Quantity': [2, 3]
})

grocery_df = pd.concat([grocery_df, new_items]).reset_index(drop=True)

item_to_drop = grocery_df[grocery_df['Item'] == 'Cheese'].index
grocery_df = grocery_df.drop(item_to_drop)  # Assign the result back to grocery_df

print(grocery_df)

This code correctly removes ‘Cheese’ from the list and assigns the updated DataFrame back to grocery_df.

输出

1
2
3
4
5
6
7

     Item  Quantity  
0    Eggs        12  
1   Bread         1  
3  Butter         1  
4    Milk         2  
5  Yogurt         3

「Pratice]

Fantastic job, top-notch analyst!
干得好，一流的分析师！

This time around, you will be adding two new items, 'Cucumber' and 'Radish', priced at $0.50 and $0.30 respectively, to our grocery list. It’s very important to update our DataFrame once the items have been added. Ready to go the extra mile? Let’s do it!
这次，您将向我们的杂货店添加两个新商品 'Cucumber' 和 'Radish' ，价格分别为 $0.50 和 $0.30 列表。添加项目后更新我们的 DataFrame 非常重要。准备好加倍努力了吗？我们开始做吧！

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import pandas as pd

  

# We will create a DataFrame for our grocery list

grocery_items = {'Items': ['Peas', 'Squash', 'Tomatoes', 'Onions'], 'Price': [1.0, 1.5, 2.5, 0.75]}

grocery_df = pd.DataFrame(grocery_items)

  

# Now let's add new items to our grocery list

  

# TODO: Create a DataFrame with two new items 'Cucumber' priced at $0.50 and 'Radish' priced at $0.30

# TODO: Concat the new items DataFrame with the current grocery DataFrame and update it

  

print(grocery_df)

Sure, here’s how you can add ‘Cucumber’ and ‘Radish’ to your grocery list and update the DataFrame:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import pandas as pd

# We will create a DataFrame for our grocery list
grocery_items = {'Items': ['Peas', 'Squash', 'Tomatoes', 'Onions'], 'Price': [1.0, 1.5, 2.5, 0.75]}
grocery_df = pd.DataFrame(grocery_items)

# Create a DataFrame with two new items 'Cucumber' priced at $0.50 and 'Radish' priced at $0.30
new_items = pd.DataFrame({
    'Items': ['Cucumber', 'Radish'],
    'Price': [0.5, 0.3]
})

# Concat the new items DataFrame with the current grocery DataFrame and update it
grocery_df = pd.concat([grocery_df, new_items]).reset_index(drop=True)

print(grocery_df)

This code creates a new DataFrame for ‘Cucumber’ and ‘Radish’ and concatenates it with the existing grocery_df. The updated DataFrame is then printed.

输出

1
2
3
4
5
6
7
8

      Items  Price  
0      Peas   1.00  
1    Squash   1.50  
2  Tomatoes   2.50  
3    Onions   0.75  
4  Cucumber   0.50  
5    Radish   0.30