Scrape Stack Overflow Jobs Using Scrapy | Python Tutorial

scrape stack overflow jobs
1. What the Hell is Scraping, Anyway?
2. Install Scrapy
3. Write Your First Script
4. Put It To Work!
For this tutorial, we’re going to write a Web Spider to scrape Stack Overflow Jobs.

1. What the Hell is Scraping, Anyway?

Web scraping (web harvesting or web data extraction) is data scraping used for extracting data from websites. Web scraping software may access the World Wide Web directly using the Hypertext Transfer Protocol, or through a web browser.

While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler.[1]

What’s a “Web Spider”?

A Web crawler, sometimes called a spider, is an Internet bot which systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).[2]

Continue reading “Scrape Stack Overflow Jobs Using Scrapy | Python Tutorial”

Python Virtual Environments

python pip python virtual environments kimberlythegeek

Why Should I Use Python Virtual Environments?

It is always recommended to use python virtual environments for all of your projects. But why?

As The Hitchhiker’s Guide to Python says:

A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them.

It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable.

Continue reading “Python Virtual Environments”

AngularJS Wikipedia Search App – Part One

Build a Wikipedia Search App With AngularJS

Part One – Get Output

In the previous post, I showed how to make unauthenticated API requests to the Wikipedia API.

In this tutorial, I’m going to rebuild the Wikipedia Search App using AngularJS.

Here’s what we’re going to start with:

// app.js

(function(){
  'use strict';

  angular.module('wikipediaApp', [])
  .constant('API_PATH', 'https://en.wikipedia.org/w/api.php')
  .controller('SearchController', SearchController);

  SearchController.$inject = ['$scope', '$http', 'API_PATH'];
  function SearchController($scope, $http, API_PATH){

  }

})();

Continue reading “AngularJS Wikipedia Search App – Part One”