2022年4月5日火曜日

Tensorflowの環境構築

2日くらいトライしてうまくいかなかったけど、やっとインストールできたっぽいので成功体験をメモ。

オフィシャルのビルドを入れようとしてみたり、Python10でやってみようとしてみたり、いろいろ試して正解にたどり着けなかったけど、やっとエラーのないインストールまでたどり着けた。Python9になっちゃったけれども。

環境

OS

  • OS: Windows11 
  • HOME Version 21H2
  • OSビルド 22000.556
  • CPU Ryzen 7 3800X

CUDA

  • cuda_11.5.2_496.13_windows.exe
  • cudnn_8.3.3.40_windows.exe

コンパイラ

  • VisualStudio2019 CommunityEdition

Python

  • python-3.9.12-amd64.exe

作業

pipのバージョン


PS > pip3.9 -V
pip 22.0.4 from D:\Python39\lib\site-packages\pip (python 3.9)

Visual Studioのインストール先


PS > (gcm cl).Definition
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\cl.exe
PS > cl
Microsoft(R) C/C++ Optimizing Compiler Version 19.29.30141 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

使い方: cl [ オプション... ] ファイル名... [ /link リンク オプション... ]
PS >
   
2019も2022もインストールしてたんだけども、なぜか2019を使うつもりがbuild中に2022を参照してしまってコケるという状況になったのでいったん2022はアンインストールして実施。 パスは通してなかったんだけどなあ。

環境変数


PS > $ENV:CUDA_PATH
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5
PS > $ENV:CUDA_PATH_V11_5
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5
PS > $ENV:CUDNN_PATH
C:\Program Files\NVIDIA\CUDNN\v8.3

インストール

    
PS > pip3.9 install -U tensorflow tf-models-official tf_slim tensorflow_datasets tensorflow-hub
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorflow
  Downloading tensorflow-2.8.0-cp39-cp39-win_amd64.whl (438.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 438.0/438.0 MB 11.5 MB/s eta 0:00:00
Collecting tf-models-official
  Downloading tf_models_official-2.8.0-py2.py3-none-any.whl (2.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 11.6 MB/s eta 0:00:00
(略)
Collecting zipp>=0.5
  Downloading zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.2.0-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.5/151.5 KB 9.4 MB/s eta 0:00:00
Using legacy 'setup.py install' for kaggle, since package 'wheel' is not installed.
Using legacy 'setup.py install' for py-cpuinfo, since package 'wheel' is not installed.
Using legacy 'setup.py install' for termcolor, since package 'wheel' is not installed.
Using legacy 'setup.py install' for promise, since package 'wheel' is not installed.
Using legacy 'setup.py install' for seqeval, since package 'wheel' is not installed.
Building wheels for collected packages: pycocotools
  Building wheel for pycocotools (pyproject.toml) ... done
  Created wheel for pycocotools: filename=pycocotools-2.0.4-cp39-cp39-win_amd64.whl size=75362 sha256=a7184aac234ded768bcf755542bfd26076d3f2d974a3b7a79b5403e294b48a59
  Stored in directory: C:\Users\xxxxx\AppData\Local\Temp\pip-ephem-wheel-cache-c_z095n_\wheels\7e\b0\8e\f2c3593944ead79f5146d057d1310ee6d7b60d30b826779846
Successfully built pycocotools
Installing collected packages: tf-estimator-nightly, text-unidecode, termcolor, tensorboard-plugin-wit, tabulate, sentencepiece, pywin32, pytz, pyasn1, py-cpuinfo, libclang, keras, gin-config, flatbuffers, certifi, zipp, wrapt, wheel, werkzeug, urllib3, uritemplate, typing-extensions, typeguard, threadpoolctl, tensorflow-io-gcs-filesystem, tensorboard-data-server, six, rsa, regex, pyyaml, python-slugify, pyparsing, pyasn1-modules, psutil, protobuf, portalocker, Pillow, oauthlib, numpy, kiwisolver, joblib, idna, gast, fonttools, dill, Cython, cycler, colorama, charset-normalizer, cachetools, tqdm, tensorflow-hub, tensorflow-addons, scipy, sacrebleu, requests, python-dateutil, promise, packaging, opt-einsum, opencv-python-headless, keras-preprocessing, importlib-metadata, httplib2, h5py, grpcio, googleapis-common-protos, google-pasta, google-auth, dm-tree, astunparse, absl-py, tf_slim, tensorflow-model-optimization, tensorflow-metadata, scikit-learn, requests-oauthlib, pandas, oauth2client, matplotlib, markdown, kaggle, google-auth-httplib2, google-api-core, tensorflow_datasets, seqeval, pycocotools, google-auth-oauthlib, google-api-python-client, tensorboard, tensorflow, tensorflow-text, tf-models-official
  Running setup.py install for termcolor ... done
  Running setup.py install for py-cpuinfo ... done
  Running setup.py install for promise ... done
  Running setup.py install for kaggle ... done
  Running setup.py install for seqeval ... done
Successfully installed Cython-0.29.28 Pillow-9.1.0 absl-py-1.0.0 astunparse-1.6.3 cachetools-5.0.0 certifi-2021.10.8 charset-normalizer-2.0.12 colorama-0.4.4 cycler-0.11.0 dill-0.3.4 dm-tree-0.1.6 flatbuffers-2.0 fonttools-4.31.2 gast-0.5.3 gin-config-0.5.0 google-api-core-2.7.1 google-api-python-client-2.42.0 google-auth-2.6.2 google-auth-httplib2-0.1.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 googleapis-common-protos-1.56.0 grpcio-1.44.0 h5py-3.6.0 httplib2-0.20.4 idna-3.3 importlib-metadata-4.11.3 joblib-1.1.0 kaggle-1.5.12 keras-2.8.0 keras-preprocessing-1.1.2 kiwisolver-1.4.2 libclang-13.0.0 markdown-3.3.6 matplotlib-3.5.1 numpy-1.22.3 oauth2client-4.1.3 oauthlib-3.2.0 opencv-python-headless-4.5.5.64 opt-einsum-3.3.0 packaging-21.3 pandas-1.4.2 portalocker-2.4.0 promise-2.3 protobuf-3.20.0 psutil-5.9.0 py-cpuinfo-8.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycocotools-2.0.4 pyparsing-3.0.7 python-dateutil-2.8.2 python-slugify-6.1.1 pytz-2022.1 pywin32-303 pyyaml-5.4.1 regex-2022.3.15 requests-2.27.1 requests-oauthlib-1.3.1 rsa-4.8 sacrebleu-2.0.0 scikit-learn-1.0.2 scipy-1.8.0 sentencepiece-0.1.96 seqeval-1.2.2 six-1.16.0 tabulate-0.8.9 tensorboard-2.8.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.8.0 tensorflow-addons-0.16.1 tensorflow-hub-0.12.0 tensorflow-io-gcs-filesystem-0.24.0 tensorflow-metadata-1.7.0 tensorflow-model-optimization-0.7.2 tensorflow-text-2.8.1 tensorflow_datasets-4.5.2 termcolor-1.1.0 text-unidecode-1.3 tf-estimator-nightly-2.8.0.dev2021122109 tf-models-official-2.8.0 tf_slim-1.1.0 threadpoolctl-3.1.0 tqdm-4.64.0 typeguard-2.13.3 typing-extensions-4.1.1 uritemplate-4.1.1 urllib3-1.26.9 werkzeug-2.1.1 wheel-0.37.1 wrapt-1.14.0 zipp-3.8.0

確認

その1


PS > pip3.9 show tensorflow
Name: tensorflow
Version: 2.8.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: d:\python39\lib\site-packages
Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, keras-preprocessing, libclang, numpy, opt-einsum, protobuf, setuptools, six, tensorboard, tensorflow-io-gcs-filesystem, termcolor, tf-estimator-nightly, typing-extensions, wrapt
Required-by: tensorflow-text, tf-models-official
PS >

その2

GPU認識しているっぽい

PS > python
Python 3.9.12 (tags/v3.9.12:b28265d, Mar 23 2022, 23:52:46) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from tensorflow.python.client import device_lib
>>> print(device_lib.list_local_devices())
2022-04-05 13:30:42.907512: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-04-05 13:30:44.780237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /device:GPU:0 with 5982 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 2070 SUPER, pci bus id: 0000:0a:00.0, compute capability: 7.5
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 3318044804479267032
xla_global_id: -1
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 6273040384
locality {
  bus_id: 1
  links {
  }
}
incarnation: 9777597053771455120
physical_device_desc: "device: 0, name: NVIDIA GeForce RTX 2070 SUPER, pci bus id: 0000:0a:00.0, compute capability: 7.5"
xla_global_id: 416903419
]
>>>