python笔记：3.2.2.9pandas数据操作_缺失值之缺失数据填充与清洗

老狼8848

593人浏览 · 2019-05-24 23:17:59

老狼8848 · 2019-05-24 23:17:59 发布

# -*- coding: utf-8 -*-
"""
Created on Fri May 24 22:49:16 2019

@author: User
"""

import pandas as pd
import numpy as np

scoresheet=pd.DataFrame({'Name':['Christoph','Morgan','Michel','Jones'],
                         'Economics':[89,97,56,82],
                         'Statistics':[98,93,76,85]})
print(scoresheet)

print("\n scoresheet['Datamining']=[79,np.nan,None,89]:")
scoresheet['Datamining']=[79,np.nan,None,89]

print(scoresheet)

print("\n scoresheet.loc[[1,3],['Name']]=[np.nan,None]:")
scoresheet.loc[[1,3],['Name']]=[np.nan,None]
print(scoresheet)

print("\n 缺失值默认不参与运算及分析:")
print(scoresheet['Datamining'].mean())


print("\n 1-----------------------:")
scoresheet['Exam_Date']=pd.date_range('20170707',periods=4)
print(scoresheet['Exam_Date'])

print("\n 2-----------------------:")
print(scoresheet)

print("\n 3-----------------------:")
scoresheet.loc[[2,3],['Exam_Date']]=np.nan
print(scoresheet)

print("\n 4-----------------------用指定数值或字符填充缺失值:")
print(scoresheet.fillna(0))

print("\n 5-----------------------")
print(scoresheet['Name'].fillna('missing'))

print("\n 6-----------------------pad表示向前填充：")
print(scoresheet.fillna(method='pad'))

print("\n 7-----------------------bfill表示向前填充：")
print(scoresheet.fillna(method='bfill'))

print("\n 8-----------------------scoresheet.bfill() 向后填充：")
print(scoresheet.bfill())

print("\n 9-----------------------limit 填充数量：")
print(scoresheet.ffill(limit=1))

print("\n 10-----------------------指定运算来填充缺失值：")
print(scoresheet['Datamining'].fillna(scoresheet['Datamining'].mean()))

print("\n 11-----------------------dropna(axis=0) 删除含有缺失值的行：")
print(scoresheet.dropna(axis=0))

print("\n 12-----------------------.dropna(how='any',axis=1) 删除含有缺失值的列：")
print(scoresheet.dropna(how='any',axis=1))

print("\n 13-----------------------")
scoresheet.loc[[0],['Exam_Date']]=np.nan
print(scoresheet)

print("\n 14-----------------------")
print(scoresheet.dropna(how='any',thresh=2,axis=1))

运行：

Name Economics Statistics
0 Christoph 89 98
1 Morgan 97 93
2 Michel 56 76
3 Jones 82 85

scoresheet['Datamining']=[79,np.nan,None,89]:
Name Economics Statistics Datamining
0 Christoph 89 98 79.0
1 Morgan 97 93 NaN
2 Michel 56 76 NaN
3 Jones 82 85 89.0

scoresheet.loc[[1,3],['Name']]=[np.nan,None]:
Name Economics Statistics Datamining
0 Christoph 89 98 79.0
1 NaN 97 93 NaN
2 Michel 56 76 NaN
3 None 82 85 89.0

缺失值默认不参与运算及分析:
84.0

1-----------------------:
0 2017-07-07
1 2017-07-08
2 2017-07-09
3 2017-07-10
Name: Exam_Date, dtype: datetime64[ns]

2-----------------------:
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07
1 NaN 97 93 NaN 2017-07-08
2 Michel 56 76 NaN 2017-07-09
3 None 82 85 89.0 2017-07-10

3-----------------------:
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07
1 NaN 97 93 NaN 2017-07-08
2 Michel 56 76 NaN NaT
3 None 82 85 89.0 NaT

4-----------------------用指定数值或字符填充缺失值:
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07 00:00:00
1 0 97 93 0.0 2017-07-08 00:00:00
2 Michel 56 76 0.0 0
3 0 82 85 89.0 0

5-----------------------
0 Christoph
1 missing
2 Michel
3 missing
Name: Name, dtype: object

6-----------------------pad表示向前填充：
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07
1 Christoph 97 93 79.0 2017-07-08
2 Michel 56 76 79.0 2017-07-08
3 Michel 82 85 89.0 2017-07-08

7-----------------------bfill表示向前填充：
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07
1 Michel 97 93 89.0 2017-07-08
2 Michel 56 76 89.0 NaT
3 None 82 85 89.0 NaT

8-----------------------scoresheet.bfill() 向后填充：
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07
1 Michel 97 93 89.0 2017-07-08
2 Michel 56 76 89.0 NaT
3 None 82 85 89.0 NaT

9-----------------------limit 填充数量：
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07
1 Christoph 97 93 79.0 2017-07-08
2 Michel 56 76 NaN 2017-07-08
3 Michel 82 85 89.0 NaT

10-----------------------指定运算来填充缺失值：
0 79.0
1 84.0
2 84.0
3 89.0
Name: Datamining, dtype: float64

11-----------------------dropna(axis=0) 删除含有缺失值的行：
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 2017-07-07

12-----------------------.dropna(how='any',axis=1) 删除含有缺失值的列：
Economics Statistics
0 89 98
1 97 93
2 56 76
3 82 85

13-----------------------
Name Economics Statistics Datamining Exam_Date
0 Christoph 89 98 79.0 NaT
1 NaN 97 93 NaN 2017-07-08
2 Michel 56 76 NaN NaT
3 None 82 85 89.0 NaT

14-----------------------
Name Economics Statistics Datamining
0 Christoph 89 98 79.0
1 NaN 97 93 NaN
2 Michel 56 76 NaN
3 None 82 85 89.0