Read and Post messages in a Slack

In this blog, we will read messages from slack and will respond to them. 

Let's first create a channel in which our bot will read the messages and then respond to them.

Create a channel:

Login to slack and in the channels, click on "Add Channels".

In this case, I am going to name the channel: slack_bot.

Now let's create an App that will read and respond to messages.

Go to Slack API. 

Click on "Create New App" and then click on "From Scratch"

Give a name to the bot. I am giving a name as "Slack Bot" and select your workspace and click on "Create App".

Under "Add Features and functionalities" select Bots.

Click on "Review Scopes to Add"

Click on "Add New Redirect URL"

Give a URL link, your company URL and if you don't have then give any other URL like Google then click on "Add" and "Save URL"

Under Scopes -> Bot Token Scopes,

Add an OAuth Scopes, select three "channels:read", "channels:history", and "chat:write".

Scroll up and click on "Install to Workspace" and click on "Allow".

This will give you Bot user OAuth Token. Copy this token and will use this to connect from Python.

Now we need to add the bot to the channel.

Go to channel (#slack_bot), and in the message type below and hit enter:

invite /@Slack Bot 

Now we are ready to interact with Slack from python.

 

import logging

import os

# Import WebClient from Python SDK 

# (github.com/slackapi/python-slack-sdk)

from slack_sdk import WebClient

from slack_sdk.errors import SlackApiError

 

# Replace the token that you copied earlier.

token_oath = 'xoxb-2222222222222-4444444444444-BPvtQxxxxxxxxxxxxxxxxxxx'

# Best practice is to get the token from env variable like

# token_oath = os.environ.get("SLACK_BOT_TOKEN")

 

# WebClient instantiates a client that can call API methods

# When using Bolt, you can use either `app.client` or the `client` passed to listeners.

client = WebClient(token=token_oath)

logging.basicConfig(filename="mylog.log")

logger = logging.getLogger(__name__)

 

# Get the channel id for slack bot

# Let's get the channedID for the channel #slack_bot

channel_name = "slack_bot"

channel_id = None

try:

    # Call the conversations.list method using the WebClient

    for result in client.conversations_list():

        if channel_id is not None:

            break

        for channel in result["channels"]:

            if channel["name"] == channel_name:

                channel_id = channel["id"]

                #Print result

                print(f"Found conversation ID: {channel_id}")

                break

 

except SlackApiError as e:

    print(f"Error: {e}")

 

# Read the conversation from channel slack bot

try:

    # Call the conversations.history method using the WebClient

    # conversations.history returns the first 100 messages by default

    # These results are paginated, 

    # see: https://api.slack.com/methods/conversations.history$pagination

    result = client.conversations_history(channel=channel_id)

 

    conversation_history = result["messages"]

 

    # Print results

    logger.info("{} messages found in {}".format(len(conversation_history), id))

 

    for message in conversation_history:

        print(message['text'])

        

except SlackApiError as e:

    logger.error("Error creating conversation: {}".format(e))

    

 

By this time, you can see all the messages. 

Now let's try to post a message once we read a message like "Hi".

for message in conversation_history:

    if message['text'] == 'Hi':

 

        # POST a message  

        channel_id = conversation_id

        try:

            # Call the conversations.list method using the WebClient

            result = client.chat_postMessage(

                channel=channel_id,

                text="Howdy!\nThis is a response from Bot."

                # You could also use a blocks[] array to send richer content

            )

            # Print result, which includes information about the message (like TS)

            print(result)

 

        except SlackApiError as e:

            print(f"Error: {e}")   

By this stage you can see the message posted on your slack channel.

Post a message in Slack

Using python, we can send automated messages to slack. In this post, we will use the webhook provided by slack to post messages in slack.

Let's follow the step-by-step process.

First, we will generate the webhook URL.

1. Log in to your slack workspace. 
2. Under apps, click on "Add Apps".
3. In the search box, write "Incoming webhook" and search.
4. Once you see "Incoming Webhook" under Available Apps, click on Add button.
5. Click on the "Add to Slack"
6. Select a channel or create a channel where the bot will post the messages.
7. Now, click on the "Add Incoming Webhooks integration" button.
8. The above step will give us a webhook URL. Copy this URL.

Now, it's time to use the webhook URL from python.

Go to your python editor and paste this code:

Make sure to update the url with the one you copied. Also, you can change the username that you will be displayed to show who has posted the message.

import json
import requests

url =  'https://hooks.slack.com/services/Txxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxxx'
headers = {'Content-Type' : 'application/json' }
slack_data = {     
"username" :  "Demo bot" ,     
"icon_emoji" :  ":satellite:" ,     
"attachments" : [
 {            
"color" : "#FF0000" ,             
"fields" :[
   {             
"title" :  "Test Message" ,
"value" :  "Hello, This is from Python" ,
"short" :  "false",
    }
 ]
  }
]
}

response = requests.post(url, data=json.dumps(slack_data), headers=headers)

print( 'Slack response' , response.status_code)

 

Once you run the code, you will see something like this on slack.

Live Streaming using Bokeh

We always wonder if we can stream the data live using python. Today, we will show how this can be achieved.
Here is what we will be able to see once we complete this article.

Let's first create a data set to read from. Here we will use a csv file that will be updated and will be read by Bokeh. This csv file will have three columns. The first column is a year and the other two columns will be random numbers generated. Create a csv file that will have data something like below and saved it in D:\Bokeh\sample_data.csv

Now let's go to our jupyter notebook and will write a code to update the csv file every one second.

from random import randint

import time

n = 1925

for x in range(n, n+50):    

  data = open('d://bokeh//sample_data.csv', "a")    

  data.writelines(f'{x},{randint(0, 100)},{randint(10, 60)}\n')    

  data.close()    

  time.sleep(1)

The above script will add the random numbers for the years 1925 to 1974. We will run this once the actual script is ready. Let's open another notebook.
To run Boken, we will need to install two libraries, install if not installed.

pip install bokeh
pip install fsspec

After installation, you can verify the version

Bokeh version 2.3.3
fsspec version 2021.07.0

After successful installation, we will import the below libraries. To update it regularly, we need to define a callback function. Here we have set it to 1s (1000 ms) to check for any update in data.

from bokeh.models import ColumnDataSource

from bokeh.plotting import figure

from bokeh.io import curdoc

import pandas as pd

#create figure

f=figure()

source_1=ColumnDataSource(data=dict(year=[],number_1=[]))

source_2=ColumnDataSource(data=dict(year=[],number_2=[]))

f.circle(x='year',y='number_1',size=7,fill_color='blue',line_color='blue',source=source_1, legend_label='Number 1')

f.line(x='year',y='number_1',source=source_1, line_color='blue')

f.circle(x='year',y='number_2',size=7,fill_color='red',line_color='red',source=source_2, legend_label='Number 2')

f.line(x='year',y='number_2',source=source_2, line_color='red')

    

curdoc().add_root(f)

curdoc().add_periodic_callback(update,1000)

On our x-axis, we will limit to show only the last 10 records by setting the value of rollover. Let's create the update function:

#create periodic function

def update():    

  data = pd.read_csv('d://bokeh//sample_data.csv')        

  x = data['year']    

  y1 = data['number_1']    

  y2 = data['number_2']        

  new_data=dict(year=x,number_1=y1)    

  source_1.stream(new_data,rollover=10)    

  new_data2=dict(year=x,number_2=y2)    

  source_2.stream(new_data2,rollover=10)  

  

The full code will look like below:

from bokeh.models import ColumnDataSource

from bokeh.plotting import figure

from bokeh.io import curdoc

import pandas as pd

#create periodic function

def update():    

  data = pd.read_csv('d://bokeh//sample_data.csv')        

  x = data['year']    

  y1 = data['number_1']    

  y2 = data['number_2']        

  new_data=dict(year=x,number_1=y1)    

  source_1.stream(new_data,rollover=10)    

  new_data2=dict(year=x,number_2=y2)    

  source_2.stream(new_data2,rollover=10)  

  
#create figure

f=figure()

source_1=ColumnDataSource(data=dict(year=[],number_1=[]))

source_2=ColumnDataSource(data=dict(year=[],number_2=[]))

f.circle(x='year',y='number_1',size=7,fill_color='blue',line_color='blue',source=source_1, legend_label='Number 1')

f.line(x='year',y='number_1',source=source_1, line_color='blue')

f.circle(x='year',y='number_2',size=7,fill_color='red',line_color='red',source=source_2, legend_label='Number 2')

f.line(x='year',y='number_2',source=source_2, line_color='red')

    

curdoc().add_root(f)

curdoc().add_periodic_callback(update,1000)

Save this as a py file. I have saved this in the same folder as our csv file. File saved as d:\bokeh\live_streaming.py
Now, we are all set to see the magic. Open a command prompt window and navigate to the folder (d:\bokeh\) and enter

python -m bokeh serve live_streaming.py

If this doesn't open the graph in the default browser, then copy the highlighted link and paste it into the browser. At this time it will show a static graph. This is now the time to update the csv file. Now run the first notebook we created to update the csv file and check the browser.
Have fun!

Date Time

Sometimes it becomes very important to use date and/or time in our code. In this session, let's try to see how we can get current date time and convert it into different formats. For this we need to import datetime module.

import datetime

current_date_time = datetime.datetime.now()

print(current_date_time)


2020-06-21 14:52:14.623809

Let's now try to get today's date or say only current day/month/year.

import datetime

today = datetime.date.today()

print("Today's Date:", today)
print("Today's day:", today.day)
print("Today's month:", today.month)
print("Today's year:", today.year)

Today's Date: 2020-06-21
Today's day: 21
Today's month: 6
Today's year: 2020

Now, let's try to convert it into different format.

import datetime

today = datetime.date.today()

print("Date in dd/mm/yyyy: ", today.strftime("%d/%m/%Y"))
print("Date in mmm dd, yyyy: ", today.strftime("%b %d, %Y"))

Date in dd/mm/yyyy:  21/06/2020
Date in mmm dd, yyyy:  Jun 21, 2020

Similarly, we can get and format time

import datetime

current_date_time = datetime.datetime.now()

print('Current Time (0-24):', current_date_time.strftime("%H:%M:%S"))
print('Current Time (0-12):', current_date_time.strftime("%I:%M:%S %p"))
print('Current Hour (0-24):', current_date_time.strftime("%H"))
print('Current Hour (0-12):', current_date_time.strftime("%I %p"))
print('Current Minute:', current_date_time.strftime("%M"))
print('Current Second:', current_date_time.strftime("%S"))


Current Time (0-24): 02:54:25
Current Time (0-12): 02:54:25 PM
Current Hour (0-24): 14
Current Hour (0-12): 14 PM
Current Minute: 54
Current Second: 25

Here are some formats that can be used

Format Code	Description	Example
%Y	YYYY	2020
%y	yy	20
%m	mm	06
%b	mmm	Jun
%B	mmmm	June
%d	dd	21
%H	0-24	14
%i	0-12	02
%p	AM/PM	PM
%M	Minute	55
%S	Seconds	32

Sets

Set is a collection of distinct elements same as we have in mathematics.

How can set help us?
If we have a list with duplicate items and we need a list with a unique item then convert the list to a set. We can also find union, intersection or difference between two sets.

Find unique elements

# Get the unique list
num_list = [1, 2, 3, 4, 2, 2, 3, 1]
num_list = set(num_list)
print(num_list)


{1, 2, 3, 4}

Add an element in the set

# Add a list of elements
new_items = [5, 6, 7, 8]
num_list.update(new_items)
print(num_list)


{1, 2, 3, 4, 5}

Add a list of elements

# Add a list of elements
new_items = [5, 6, 7, 8]
num_list.update(new_items)
print(num_list)


{1, 2, 3, 4, 5, 6, 7, 8}

Now, let's perform some basic operation on two sets such as union, intersection and difference of these sets.

Union

# union of two sets
set_a = {1, 2, 3, 4}
set_b = {2, 4, 6, 8}
print(set_a.union(set_b))


{1, 2, 3, 4, 6, 8}

Intersection

# intersection of two sets
set_a = {1, 2, 3, 4}
set_b = {2, 4, 6, 8}
print(set_a.intersection(set_b))


{2, 4}

Difference

# difference of two sets
set_a = {1, 2, 3, 4}
set_b = {2, 4, 6, 8}
print(set_a - set_b)


{1, 3}

Let's try to find out if a set is subset or superset of a given set.
Subset

# subset
set_1 = {1, 3, 5}
set_2 = {1, 2, 3, 4, 5, 6}
set_3 = {1, 3, 5, 7}
print('set_1 is subset of set_2:', set_1.issubset(set_2))
print('set_3 is subset of set_2:', set_3.issubset(set_2))


set_1 is subset of set_2: True
set_3 is subset of set_2: False

Superset

# superset and subset
set_1 = {1, 3, 5}
set_2 = {1, 2, 3, 4, 5, 6}
set_3 = {1, 3, 5, 7}
print('set_3 is superset of set_2:', set_3.issuperset(set_2))


set_3 is superset of set_2: False

Data Analysis for new store location

ABC Bookstore

1. Introduction

As Nelson Mandela says "Education is the most powerful weapon which you can use to change the world". Education not only makes a person knowledgeable but also helps them grow and grab better opportunities in their life to be successful. It also makes an individual more responsible, take the right decisions and develop them personally and socially. It also helps one to do their daily activities in the best possible ways. It helps in acquiring knowledge and new skills that will impact their personal development and for their country. To study, one needs to have access to good resources of books and a book store. 

2. Business Problem

ABC company is located in New York, is a known and independent book store focused on staff curated book selections. Stores are generally located near a school mainly helping students with academic books and stationaries. The bookstore will offer a wide selection of books including children’s literature, modern fiction, true crime, cookbooks, foreign language titles and art books. The cosy bookstore will also offer a book club, author events and children’s story hour and gift-wrapping section. ABC company is looking to help students and community by opening a new branch in Sydney, Australia.

To open a bookstore, a few things that will be considered while finalizing the location. Below are the important things that need to be taken into consideration:

1.     The bookstore should be located within a 2 km radius of a school.

2.     The school rating should be 7 or above out of 10.

3.     There should not be more than 1 bookstore within a 2 km radius of the school.

4.     There should be a bank/atm in the vicinity.

3. Data Location Gathering:

In Sydney, there are a lot of schools but very limited bookstore available. In order to finalize an optimum location, data from Foursquare.com will be gathered. It will also help us grab the rating for different schools along with the number of bookstores or bank available in its approximation areas. Foursquare has rich information available and above all, all of this information can be extracted by using their API service. This API service will gather a list of all the schools available in Sydney along with their rating. Once all the schools' location (latitude and longitude) and filtered with rating 7 or above, we will look for other criteria such as the number of other bookstores and bank/atm availability.

In order to generate the best location, we would need to find

·       schools in Sydney

·       schools average rating

·       number of existing book stores around the school

·       other facilities such as bank/atm

We will look for schools within a radius of 30 Km from Sydney. For the existing bookstore or bank/atm we will search within 2 km of radius from school.

4. Data Acquisition and cleaning

4.1  Data Sources

Retrieving the data is the most important aspect of any data analysis. For our analysis, we used Foursquare API to pull a list of all the schools and list of all book stores, banks and ATM’s near each school that was required for the data analysis. There are various sources that provide the data but either they do not have any API service to use or have a very limited number of calls we can make through their website for free. Foursquare helped us by letting us scrap most of the data in the first couple of days.

4.2  Data Cleaning

Data scrapped from various sources were combined, cleaned and stored in various dataframe for analysis. There were several problems with the data available.

Firstly, there were missing data that we couldn’t use, hence needs to be dropped.

Secondly, the datatypes were converted to the proper format

Thirdly, some of them were having null values that were replaced will 0.

4.3  Data Selection

While considering the data, we need to make sure we use the data from the correct location. For that, we used a radius of 30 km of Sydney location. It retrieved 100 schools. Later, we also need to retrieve their average rating. And finally, we need to pull a list of all book stores, banks, atm within 2 km radius of each school.

5. Exploratory Data Analysis

Once we retrieved and data was pre-processed, we plotted all our schools, book stores, banks and atm’s on the map to visualize it further (Figure 1).

Fig 1: Location of all 100 schools, ~800 book stores and ~400 Banks/ATM’s in Sydney

This image has around 800 book stores, 400 banks/atm and 100 schools.

There were few places which have a staggering number of book stores within 2 km of these schools. Some of the schools have 35 book stores around them. This was again plotted to see where these were located. There were 8 schools around which more than 30 book stores were present within its 2 km radius (Figure 2).

Fig 2: Location of 8 schools having 30+ book store existing around each school in Sydney

Finally, after sorting and filtering our data, we were able to gather our prime candidates with least number of book stores in its vicinity and have at least 1 bank/atm with its 2 km radius. Figure 3 shows the locations of these schools.

Fig 3: Location of 7 schools having 1 book store and at least 1 bank/atm around each school in Sydney

6. Results and Discussions

Our results show that although we had 100 schools with around 800 books store and approximately 400 atm’s/banks within a radius of 2 km from the schools, there are few schools where a number of books store was minimal. There were few schools around which a staggering number of books stores existed (35 book stores). Most of these schools were located in the Central/Financial district of Sydney. Our potential location candidate for book stores was in the suburbs of Sydney.

With narrowing down all the available schools and as per the requirement, there were 7 schools that have at least 1 atm/bank and at max 1 book store within its vicinity. These schools are located in:

1.     one school in the upper north suburb of Sydney

2.     two schools in the western suburb of Sydney

3.     two schools in the south-west suburb of Sydney and

4.     two schools in the eastern suburb of Sydney

7. Conclusions

Purpose of this project was to identify an optimal location for opening a book store with the least number of bookstores around a school and have at least one bank or atm near it. Identifying these schools, existing bank stores and banks or ATM's according to with the help for Foursquare API's has helped achieve our requirement by the exploration of these data points and can be shared with the stakeholders.

Based on the results, a stakeholder can take an appropriate decision based on other specific characteristics such as real estate price and availability, number of people living in that neighbourhood, accessibility to parking, public transportation in each recommended zone. They can also take into considerations like accessibility to highway or types of lease available for the shop.