![](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/04\/Medium-Blog.png\" "\\"Medium")
<\/span><\/div>![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/Antoine-Aubay-300x300.jpeg\")
A step by step guide on how to collect and preprocess satellite imagery data for machine learning algorithms<\/h3><\/div>This article is the first part of a 2 part series which explores the uses of machine learning algorithms on satellite imagery. Here we will focus on the data collection and preprocessing steps while the second part will showcase the use of various machine learning algorithms.<\/em><\/h4><\/div><\/div><\/div><\/div><\/article>TL;DR<\/strong>
\nThis article will:<\/p>\nGive you a better understanding of satellite imagery\n Show you what type of imagery can be collected open source\n Guide you through the necessary steps to have \u201cmachine learning ready\u201d images\nWe will use the detection and classification of agriculture fields as an example. Python and jupyter notebook are used for the preprocessing steps. The jupyter notebook for reproducing the steps is available in github<\/a>.
\nThis article assumes basic fundamentals in data science and python.<\/p>\n<\/div><\/div><\/div><\/div><\/div>Computer vision is a fascinating branch of machine learning that has been quickly developing in past years. With public access to many off the shelf algorithm libraries (Tensorflow<\/a>, OpenCv<\/a>, Pytorch<\/a>, Fastai<\/a> \u2026) as well as open source datasets (CIFAR -10<\/a>, Imagenet<\/a>, IMDB<\/a> \u2026) it\u2019s very easy for a data scientist to get hands on experience on this topic.<\/h4><\/div>But did you know that you could also access open source satellite imagery ? Those types of imagery can be used in a wealth of use cases (Agriculture, Logistics, Energy \u2026). However, they provide additional challenges for a data scientist due to their size and complexity. I have written this article to share some of my key learnings working with them.<\/h4><\/div>Step 1 \u2014 Select the satellite open data source<\/h2><\/div>First, you need to choose which satellite you want to use. Key features to watch out for here are :<\/p>\n<\/div>
- <\/i><\/span>\n
Spatial Resolution<\/strong>: how many meters are covered per pixel, higher resolution will unlock some use cases and give better performance for most algorithms.<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Instruments available<\/strong>: the instruments will capture different spectral bands, both from the visible and invisible spectrum. Bands usefulness will vary depending on the use case.<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Temporal resolution<\/strong> : time between two visits on a given spot on earth. Note that many satellite systems actually include several satellites and in that case the global temporal resolution is usually equal to that of one of its satellites divided by the number of satellites.<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Radiometric resolution<\/strong> : the number of possible values the instrument captures. The higher, the more precise the measurements are.<\/p>\n<\/div><\/li><\/ul>Here are satellites whose data are available free of charge :<\/p>\n<\/div>
![\"\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/Benchmark-of-satellite-systems.png\")
<\/p>\n
Benchmark of satellite systems<\/p>\n<\/div>
Since our goal is to detect and identify agricultural crops, the visible spectrum (which is used to provide standard color images) should be quite useful. Additionally there are correlations between nitrogen level of crops and NIR (Near-infrared) spectral bands. As far as free options go, Sentinel 2 is currently the best in terms of resolution with 10 m for those spectral bands, so we are going to use this satellite.<\/p>\n<\/div>
Of course, 10 m resolution is much lower compared to state of the art in satellite imagery with commercial satellites reaching a 30 cm resolution. Access to commercial satellite data can unlock many use cases, but despite increasing competition with new players entering this market, they remain highly priced and hence difficult to integrate in a scaled use case.<\/p>\n<\/div>
Step 2 \u2014 Collect the satellite imagery data from Sentinel 2<\/h2><\/div>Now that we have selected our satellite source, the new step is to download the images. Sentinel 2 is part of the Copernicus Programme from the European Space Agency<\/a>. We can access its data using the SentinelSat python API<\/a>.<\/p>\n<\/div>We first need to create an account at the Copernicus Open Access Hub<\/a>. Alternatively note that this website also provides a way to query images with a GUI.<\/p>\n<\/div>A way to specify the zones you want to explore is to provide the API with a geojson file, which is a json containing GPS coordinates position (latitude and longitude) of a zone. Geojson can be quickly created on this website<\/a>.<\/p>\n<\/div>This way you can query all the Satellite images intersecting with the zone provided.<\/p>\n<\/div>
Here we list all images available on a for a given zone and time range :<\/p>\n<\/div>
\napi = SentinelAPI(
\ncredentials[“username”],
\ncredentials[“password”],
\n“https:\/\/scihub.copernicus.eu\/dhus”
\n)<\/p>\n
shape = geojson_to_wkt(read_geojson(geojson_path))<\/p>\n
images = api.query(
\nshape,
\ndate=(date(2020, 5, 1), date(2020, 5, 10)),
\nplatformname=”Sentinel-2″,
\nprocessinglevel = “Level-2A”,
\ncloudcoverpercentage=(0, 30)
\n)<\/p>\n
images_df = api.to_dataframe(images)<\/p>\n<\/div>\n<\/div>
Note the use of arguments:<\/p>\n<\/div>
This article is the first part of a 2 part series which explores the uses of machine learning algorithms on satellite imagery. Here we will focus on the data collection and preprocessing steps while the second part will showcase the use of various machine learning algorithms.<\/em><\/h4><\/div><\/div><\/div><\/div><\/article>TL;DR<\/strong>
\nThis article will:<\/p>\nGive you a better understanding of satellite imagery\n Show you what type of imagery can be collected open source\n Guide you through the necessary steps to have \u201cmachine learning ready\u201d images\nWe will use the detection and classification of agriculture fields as an example. Python and jupyter notebook are used for the preprocessing steps. The jupyter notebook for reproducing the steps is available in github<\/a>.
\nThis article assumes basic fundamentals in data science and python.<\/p>\n<\/div><\/div><\/div><\/div><\/div>Computer vision is a fascinating branch of machine learning that has been quickly developing in past years. With public access to many off the shelf algorithm libraries (Tensorflow<\/a>, OpenCv<\/a>, Pytorch<\/a>, Fastai<\/a> \u2026) as well as open source datasets (CIFAR -10<\/a>, Imagenet<\/a>, IMDB<\/a> \u2026) it\u2019s very easy for a data scientist to get hands on experience on this topic.<\/h4><\/div>But did you know that you could also access open source satellite imagery ? Those types of imagery can be used in a wealth of use cases (Agriculture, Logistics, Energy \u2026). However, they provide additional challenges for a data scientist due to their size and complexity. I have written this article to share some of my key learnings working with them.<\/h4><\/div>Step 1 \u2014 Select the satellite open data source<\/h2><\/div>First, you need to choose which satellite you want to use. Key features to watch out for here are :<\/p>\n<\/div>
- <\/i><\/span>\n
Spatial Resolution<\/strong>: how many meters are covered per pixel, higher resolution will unlock some use cases and give better performance for most algorithms.<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Instruments available<\/strong>: the instruments will capture different spectral bands, both from the visible and invisible spectrum. Bands usefulness will vary depending on the use case.<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Temporal resolution<\/strong> : time between two visits on a given spot on earth. Note that many satellite systems actually include several satellites and in that case the global temporal resolution is usually equal to that of one of its satellites divided by the number of satellites.<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Radiometric resolution<\/strong> : the number of possible values the instrument captures. The higher, the more precise the measurements are.<\/p>\n<\/div><\/li><\/ul>Here are satellites whose data are available free of charge :<\/p>\n<\/div>
<\/p>\n
Benchmark of satellite systems<\/p>\n<\/div>
Since our goal is to detect and identify agricultural crops, the visible spectrum (which is used to provide standard color images) should be quite useful. Additionally there are correlations between nitrogen level of crops and NIR (Near-infrared) spectral bands. As far as free options go, Sentinel 2 is currently the best in terms of resolution with 10 m for those spectral bands, so we are going to use this satellite.<\/p>\n<\/div>
Of course, 10 m resolution is much lower compared to state of the art in satellite imagery with commercial satellites reaching a 30 cm resolution. Access to commercial satellite data can unlock many use cases, but despite increasing competition with new players entering this market, they remain highly priced and hence difficult to integrate in a scaled use case.<\/p>\n<\/div>
Step 2 \u2014 Collect the satellite imagery data from Sentinel 2<\/h2><\/div>Now that we have selected our satellite source, the new step is to download the images. Sentinel 2 is part of the Copernicus Programme from the European Space Agency<\/a>. We can access its data using the SentinelSat python API<\/a>.<\/p>\n<\/div>We first need to create an account at the Copernicus Open Access Hub<\/a>. Alternatively note that this website also provides a way to query images with a GUI.<\/p>\n<\/div>A way to specify the zones you want to explore is to provide the API with a geojson file, which is a json containing GPS coordinates position (latitude and longitude) of a zone. Geojson can be quickly created on this website<\/a>.<\/p>\n<\/div>This way you can query all the Satellite images intersecting with the zone provided.<\/p>\n<\/div>
Here we list all images available on a for a given zone and time range :<\/p>\n<\/div>
\napi = SentinelAPI(
\ncredentials[“username”],
\ncredentials[“password”],
\n“https:\/\/scihub.copernicus.eu\/dhus”
\n)<\/p>\n
shape = geojson_to_wkt(read_geojson(geojson_path))<\/p>\n
images = api.query(
\nshape,
\ndate=(date(2020, 5, 1), date(2020, 5, 10)),
\nplatformname=”Sentinel-2″,
\nprocessinglevel = “Level-2A”,
\ncloudcoverpercentage=(0, 30)
\n)<\/p>\n
images_df = api.to_dataframe(images)<\/p>\n<\/div>\n<\/div>
Note the use of arguments:<\/p>\n<\/div>
TL;DR<\/strong> We will use the detection and classification of agriculture fields as an example. Python and jupyter notebook are used for the preprocessing steps. The jupyter notebook for reproducing the steps is available in github<\/a>. First, you need to choose which satellite you want to use. Key features to watch out for here are :<\/p>\n<\/div> Spatial Resolution<\/strong>: how many meters are covered per pixel, higher resolution will unlock some use cases and give better performance for most algorithms.<\/p>\n<\/div><\/li> Instruments available<\/strong>: the instruments will capture different spectral bands, both from the visible and invisible spectrum. Bands usefulness will vary depending on the use case.<\/p>\n<\/div><\/li> Temporal resolution<\/strong> : time between two visits on a given spot on earth. Note that many satellite systems actually include several satellites and in that case the global temporal resolution is usually equal to that of one of its satellites divided by the number of satellites.<\/p>\n<\/div><\/li> Radiometric resolution<\/strong> : the number of possible values the instrument captures. The higher, the more precise the measurements are.<\/p>\n<\/div><\/li><\/ul> Here are satellites whose data are available free of charge :<\/p>\n<\/div> Benchmark of satellite systems<\/p>\n<\/div> Since our goal is to detect and identify agricultural crops, the visible spectrum (which is used to provide standard color images) should be quite useful. Additionally there are correlations between nitrogen level of crops and NIR (Near-infrared) spectral bands. As far as free options go, Sentinel 2 is currently the best in terms of resolution with 10 m for those spectral bands, so we are going to use this satellite.<\/p>\n<\/div> Of course, 10 m resolution is much lower compared to state of the art in satellite imagery with commercial satellites reaching a 30 cm resolution. Access to commercial satellite data can unlock many use cases, but despite increasing competition with new players entering this market, they remain highly priced and hence difficult to integrate in a scaled use case.<\/p>\n<\/div> Now that we have selected our satellite source, the new step is to download the images. Sentinel 2 is part of the Copernicus Programme from the European Space Agency<\/a>. We can access its data using the SentinelSat python API<\/a>.<\/p>\n<\/div> We first need to create an account at the Copernicus Open Access Hub<\/a>. Alternatively note that this website also provides a way to query images with a GUI.<\/p>\n<\/div> A way to specify the zones you want to explore is to provide the API with a geojson file, which is a json containing GPS coordinates position (latitude and longitude) of a zone. Geojson can be quickly created on this website<\/a>.<\/p>\n<\/div> This way you can query all the Satellite images intersecting with the zone provided.<\/p>\n<\/div> Here we list all images available on a for a given zone and time range :<\/p>\n<\/div> api = SentinelAPI( shape = geojson_to_wkt(read_geojson(geojson_path))<\/p>\n images = api.query( images_df = api.to_dataframe(images)<\/p>\n<\/div>\n<\/div> Note the use of arguments:<\/p>\n<\/div>
\nThis article will:<\/p>\n
\nThis article assumes basic fundamentals in data science and python.<\/p>\n<\/div><\/div><\/div><\/div><\/div>Computer vision is a fascinating branch of machine learning that has been quickly developing in past years. With public access to many off the shelf algorithm libraries (Tensorflow<\/a>, OpenCv<\/a>, Pytorch<\/a>, Fastai<\/a> \u2026) as well as open source datasets (CIFAR -10<\/a>, Imagenet<\/a>, IMDB<\/a> \u2026) it\u2019s very easy for a data scientist to get hands on experience on this topic.<\/h4><\/div>
But did you know that you could also access open source satellite imagery ? Those types of imagery can be used in a wealth of use cases (Agriculture, Logistics, Energy \u2026). However, they provide additional challenges for a data scientist due to their size and complexity. I have written this article to share some of my key learnings working with them.<\/h4><\/div>
Step 1 \u2014 Select the satellite open data source<\/h2><\/div>
<\/p>\nStep 2 \u2014 Collect the satellite imagery data from Sentinel 2<\/h2><\/div>
\ncredentials[“username”],
\ncredentials[“password”],
\n“https:\/\/scihub.copernicus.eu\/dhus”
\n)<\/p>\n
\nshape,
\ndate=(date(2020, 5, 1), date(2020, 5, 10)),
\nplatformname=”Sentinel-2″,
\nprocessinglevel = “Level-2A”,
\ncloudcoverpercentage=(0, 30)
\n)<\/p>\n
![\"\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/images_df.png\")
<\/p>\n![\"\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/Satellite-Image-\u2014-Color-intensity-issue-Copernicus-Sentinel-data-2020-.png\")
<\/p>\n![\"\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/Satellite-Image-\u2014-Right-color-intensity-Copernicus-Sentinel-data-2020.png\")
<\/p>\n![\"\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/Fragment-of-the-satellite-image-Copernicus-Sentinel-data-2020.png\")
<\/p>\n![\"\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/05\/Globe.jpeg\")
<\/p>\n<\/div>