FIU Libraries: Data Processing & Preparation: Open Source Methods

Methods

Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. Python's simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance. Python supports modules and packages, which encourages program modularity and code reuse. The Python interpreter and the extensive standard library are available in source or binary form without charge for all major platforms, and can be freely distributed.

Library

Scipy library is one of the core packages that make up the SciPy stack. It provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization

Numpy is the fundamental package for scientific computing with Python. It contains among other things:

a powerful N-dimensional array object
sophisticated (broadcasting) functions
tools for integrating C/C++ and Fortran code
useful linear algebra, Fourier transform, and random number capabilities

Get started here with Numpy and Scipy.

Integrated development environments for Python

Atom, an open source cross-platform IDE with autocomplete, help and more Python features under package extensions.
Boa Constructor, a cross-platform IDE for Python development
EasyEclipse, an open source IDE for Python and other languages
Eclipse with the Pydev plug-in. Eclipse supports many other languages as well.
Eric, an IDE for Python and Ruby
Geany, IDE for Python development and other languages
Koding a free online development environment for many programming languages including Python.
Komodo IDE an IDE for Python, Perl, PHP and Ruby
NetBeans, is written in Java and runs everywhere where a JVM is installed.
Ninja-IDE, free software, written in Python and Qt, Ninja name stands for Ninja-IDE Is Not Just Another IDE
PIDA, open source IDE written in Python capable of embedding other text editors, such as Vim
PyCharm, a proprietary and Open Source IDE for Python development
PyScripter, Free and open-source software Python IDE for Windows.
PythonAnywhere, an online IDE and Web hosting service
Python Tools for Visual Studio, Free and open-source plugin or Visual Studio.
Stani's Python Editor (SPE), a cross-platform IDE for Python development
Sublime Text, a text editor.
Webware for Python, a suite of programming tools for constructing web-based applications in Python
Wing IDE, cross-platform proprietary with some free versions/licenses IDE for Python

Packages

ArcPY: ArcPy is a Python site package that provides a useful and productive way to perform geographic data analysis, data conversion, data management, and map automation with Python into ArcGIS. ArcGIS applications written with ArcPy benefit from the development of additional modules in numerous niches of Python by GIS professionals and programmers from many different disciplines.

R is ‘GNU S’, a freely available language and environment for statistical computing and graphics which provides a wide variety of statistical and graphical techniques: linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. It compiles and runs on a wide variety of UNIX platforms, Windows and MacOS.

Packages

Packages are the fundamental units of reproducible R code. They include reusable R functions, the documentation that describes how to use them, and sample data. Depending on the analisys you want to apply, you can use a different package.

A list of packages can be found here

Integrated development environments (IDE) for R

R Studio that includes console, syntax-highlighting editor, as well as tools for plotting, history, debugging and workspace management.
Eclipse /StatET it is a eclipse based IDE for R. If offers a set of tools for R coding and package building.
Visual Studio for R

There are many GIS/RS open source software to process geospatial data.

Learn about some free GIS/RS software that doesn't require development skill to process your data.

More information about software and getting started on the "Software" session.

QGIS
QGIS is jam-packed with hidden gems at your fingertips. For example, you can automate map production, process geospatial data, and generate drool-worthy cartographic figures. There’s no other free mapping software on this list that lets you map like a rock star than QGIS.
gVSIG
In 2004, the gvSIG project emerged as a free, open source GIS software option in Spain. We illustrate in this gvSIG guide and review why we like it SO much: gvSIG really outperforms QGIS 2 for 3D. It really is the best 3D visualization available in open source GIS.
Whitebox GAT
The Whitebox GAT project began in 2009 and was conceived as a replacement for the Terrain Analysis System (TAS). Whitebox GAT was intended to have a broader focus than its predecessor, positioning it as an open-source desktop GIS and remote sensing software package for general applications of geospatial analysis and data visualization
ILWIS
Free GIS software users rejoice. Once commercial GIS software, now turned into open source GIS. ILWIS (Integrated Land and Water Information Management) is an oldie but a goodie.
SAGA
SAGA GIS (System for Automated Geoscientific Analyses) is one of the classics in the world of free GIS software. It started out primarily for terrain analysis such as hillshading, watershed extraction and visibility analysis.
GRASS GIS
GRASS GIS (Geographic Resources Analysis Support System) was developed by the US Army Corps of Engineers as a tool for land management and environmental planning. It has over 350 vector and raster manipulation tools.

Python Parallel Programming
Offered through Linda.com, you can learn parallel programming techniques using Python and explore the many ways you can write code that allows more than one task to occur at a time. First, discover how to develop and implement efficient software architecture that is set up to take advantage of thread-based and process-based parallelism. Next, find out how to use Python modules for asynchronous programming. Then, explore GPU programming using PyCUDA, NumbaPro, and PyOpenCL. This course provides extensive coverage of synchronizing processes, streamlining communication, reducing operations, and optimizing code so you can select and implement the right parallel processing solutions for your applications.
Stackoverflow
Learn, Share, Build - Each month, over 50 million developers come to Stack Overflow to learn, share their knowledge, and build their careers.
DataCamp
Learn R, Python & Data Science Online | DataCamp
FIU GIS Center Github
Some scripts in R.

Floor		Resources
One		Center for Academic Success: University Learning/Testing Center Electronic Classroom Broadcast Video Productions Resource for Education Technology UTS - Media Equipment Services Distance Learning Electronic Classroom Instructional Photography & Graphics
Two		Information & Research Services Curriculum & Juvenile Collection LACiC - Latin American and Caribbean Center Government Documents GIS-RS Center & Teaching Lab Maps & Imagery User Statistics
Three		Periodicals & Newspapers Laptop Lounge Honors College Student Resource Center College of Medicine Library Copiers Systems
Four		College of Medicine Special Collections & Archivese Academic Space Management School of Social Work Study Area
Five		Audiovisual Collection Diaz Ayala Collection Architecture and Fine Arts (General Collection and Periodical Section N) Music (General Collection and Periodical Section M) Music Listening Room Music CAI Center Spanish Resource Center Group Study Rooms
Six		Quiet Study Space Group Study Rooms General Collection HJ-L & P-Z Oversized Books
Seven		General Collection A-HG Quiet Study Floor Group Study Rooms
Eight		Library Administration Collection Management Digital Collections Center

Floor		Resources
One		Circulation Course Reserves Intercampus/Interlibrary Loan Information and Research Services Center for Excellence in Writing Communication Arts Studio Computer Labs/Copiers
Two		Microforms Periodicals Juvenile Collection Oversize Collection Video Collection Government Documents Reference Workroom Reference Collection Copiers
Three		General Collection, A-Z Library Administration

Data Processing & Preparation

Methods

Packages

Integrated development environments (IDE) for R

Information

Libraries

Contact