项目简介
本项目使用paddle的动态图机制实现了经典的图像分类网络：AlexNet，并在公开的蔬菜数据集上进行了模型训练以及验证，建议使用GPU来运行本项目。静态图版本请查看：用PaddlePaddle实现图像分类-AlexNet
下载安装命令
## CPU版本安装命令
pip install -f https://paddlepaddle.org.cn/pip/oschina/cpu paddlepaddle
## GPU版本安装命令
pip install -f https://paddlepaddle.org.cn/pip/oschina/gpu paddlepaddle-gpu
数据集介绍
解压蔬菜数据集，里面包括三个目录，分别是三个类别对应的数据，具体的数据预处理过程见后续介绍
网络结构介绍
AlexNet网络具体结构如下图所示：
原论文中是采取了两块GPU进行交互，所以在图中有两条支路。AlexNet架构有6000万参数和650000个神经元，包含5层卷积网络，其中一些含有max pooling，还有3层全连接层，最后一层的节点数1000个，采用softmax分类
参考链接：AlexNet学习
论文原文：ImageNet Classification with Deep ConvolutionalNeural Networks
In[1]
# 解压蔬菜数据集
!cd data/data504 && unzip -q vegetables.zip
解压预训练参数，去掉了最后分类的全连接层
预处理数据，将其转化为标准格式。同时将数据拆分成两份，以便训练和计算预估准确率
- label_list.txt 每一行一个类别，类别编号/t类别名字
- train.txt 每一行一个样本，图片路径/t类别编号
- trainImageSet/xxx.jpg 训练图片
- eval.txt 格式通 train.txt
- evalImageSet/xxx.jpg 验证图片
In[2]
import codecs
import os
import random
import shutil
from PIL import Image
train_ratio = 4.0 / 5
all_file_dir = 'data/data504/vegetables'
class_list = [c for c in os.listdir(all_file_dir) if os.path.isdir(os.path.join(all_file_dir, c)) and not c.endswith('Set') and not c.startswith('.')]
class_list.sort()
print(class_list)
train_image_dir = os.path.join(all_file_dir, "trainImageSet")
if not os.path.exists(train_image_dir):
os.makedirs(train_image_dir)
eval_image_dir = os.path.join(all_file_dir, "evalImageSet")
if not os.path.exists(eval_image_dir):
os.makedirs(eval_image_dir)
train_file = codecs.open(os.path.join(all_file_dir, "train.txt"), 'w')
eval_file = codecs.open(os.path.join(all_file_dir, "eval.txt"), 'w')
with codecs.open(os.path.join(all_file_dir, "label_list.txt"), "w") as label_list:
label_id = 0
for class_dir in class_list:
label_list.write("{0}/t{1}/n".format(label_id, class_dir))
image_path_pre = os.path.join(all_file_dir, class_dir)
for file in os.listdir(image_path_pre):
try:
img = Image.open(os.path.join(image_path_pre, file))
if random.uniform(0, 1) <= train_ratio:
shutil.copyfile(os.path.join(image_path_pre, file), os.path.join(train_image_dir, file))
train_file.write("{0}/t{1}/n".format(os.path.join(train_image_dir, file), label_id))
else:
shutil.copyfile(os.path.join(image_path_pre, file), os.path.join(eval_image_dir, file))
eval_file.write("{0}/t{1}/n".format(os.path.join(eval_image_dir, file), label_id))
except Exception as e:
pass
# 存在一些文件打不开，此处需要稍作清洗
label_id += 1
train_file.close()
eval_file.close()['cuke', 'lettuce', 'lotus_root']
In[5]
!python work/train.pyW0306 15:44:50.494469 222 device_context.cc:237] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.0 W0306 15:44:50.498114 222 device_context.cc:245] device: 0, cuDNN Version: 7.3. 2020-03-06 15:44:52,723 - train.py[line:76] - INFO: Loss at epoch 0 step 0: [1.2393346], acc: [0.328125] 2020-03-06 15:44:58,777 - train.py[line:76] - INFO: Loss at epoch 1 step 0: [1.1206189], acc: [0.46875] 2020-03-06 15:45:04,972 - train.py[line:76] - INFO: Loss at epoch 2 step 0: [1.2411859], acc: [0.390625] 2020-03-06 15:45:11,130 - train.py[line:76] - INFO: Loss at epoch 3 step 0: [0.78164065], acc: [0.671875] 2020-03-06 15:45:17,355 - train.py[line:76] - INFO: Loss at epoch 4 step 0: [0.5699992], acc: [0.734375] 2020-03-06 15:45:23,380 - train.py[line:76] - INFO: Loss at epoch 5 step 0: [0.42954433], acc: [0.859375] 2020-03-06 15:45:29,459 - train.py[line:76] - INFO: Loss at epoch 6 step 0: [0.25614244], acc: [0.890625] 2020-03-06 15:45:35,796 - train.py[line:76] - INFO: Loss at epoch 7 step 0: [0.75625485], acc: [0.765625] 2020-03-06 15:45:41,868 - train.py[line:76] - INFO: Loss at epoch 8 step 0: [0.45293224], acc: [0.78125] 2020-03-06 15:45:48,243 - train.py[line:76] - INFO: Loss at epoch 9 step 0: [0.34878293], acc: [0.859375] 2020-03-06 15:45:53,480 - train.py[line:78] - INFO: Final loss: [0.26190406]
In[6]
!python work/eval.pyW0306 15:49:41.288347 282 device_context.cc:237] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.0 W0306 15:49:41.291836 282 device_context.cc:245] device: 0, cuDNN Version: 7.3. 0.9137931
点击链接，使用AI Studio一键上手实践项目吧：https://aistudio.baidu.com/aistudio/projectdetail/169433
下载安装命令 ## CPU版本安装命令 pip install -f https://paddlepaddle.org.cn/pip/oschina/cpu paddlepaddle ## GPU版本安装命令 pip install -f https://paddlepaddle.org.cn/pip/oschina/gpu paddlepaddle-gpu
>> 访问 PaddlePaddle 官网，了解更多相关内容。
{{o.name}}
{{m.name}}
原创文章，作者：ItWorker，如若转载，请注明出处：https://blog.ytso.com/72693.html