# Posts

## Market Profile Chart using Python!

Continuing from the earlier code  where I created the profile data using 'Alphabets, here let me try to plot the chart. This one turned out bit tricky and used an example snippet from Matplotlib for stacked bar chart!Logic is simple, instead of Alphabets, have an a number for each occurrence, use Matplotlib stacked bar chart to plot the colors.mi=defaultdict(list) TGroups=df.groupby([pd.Grouper(key='DateTime', freq=frequency)]) #iterate over each group and add to dictionary, #dictonary keys are 'High' and 'Low' of each group, #rounded and values are char A, B incremented for each period(freq group) #default 30 min since we grouped based on freq, #for each group increment the char i.e +1 min_price=np.round(df.min().Low) max_price=np.round(df.max().High) for t,g in TGroups: g_min_price=np.round(g.Low.min()) g_max_price=np.round(g.High.max()) for price in range(int(min_price), int(max_price+1)): if (price > g_max_price) or (price < g_min_price): mi[price].append(0) else:…

## Market Profile using Python!

If anyone did a technical analysis using popular trading charts, would have noticed Market Profile as one of the techniques. This was a really different way of looking at the time series data.The logic is slice the data for a period, say every '30 min' and assign an alphabet, starting with 'A' to all the 'Price'. Price is also rounded of so that we don't have too many keys based on Price. Simply put in programming parlance create a dictionary with Price(rounded of) as keys and add new Alphabet for every slice.Say first 30 min, the low is 100 and high is 110, then starting with 100 to 110, append 'A' as a value. so our dictionary would be[{100, 'A'},{101, 'A'},{102, 'A'},{103, 'A'} ..{110,'A'}],then, next  slice of 30 min, the…

Logistic is  a classification algorithm based on probability, instead of predicting the class, it gives the log likelihood of being the given class i.e (p(y/x)  = p/(1-p))Let's look at a little bit of theory about it. We use the same linear equation only difference is, the y is binary, that is  either 0 or 1.[h_{theta}(x) =  theta_{0} + theta_{1}x]However we calculate the y using the sigmoid function to return a probability value between 0 and 1. [P(class=1) = frac{1} {1 + e^{-z}}] The reason this function returns 0 or 1 for any large positive or negative and the z would be[h_{theta}(x) = 1 / (1 + e^{-z}), z = theta_{0} + theta_{1}x]Cost function[J(theta) = sum_{i=1}^{n}y_{i}log(h_{theta}(x_{i})) + (1-y_{i})log(1-h_{theta}(x_{i}))]Here is a complete derivation, a nice one, lecture notes on logistic regressionUpdate rule Same as…

## It all starts with y=mx+b

It all starts with the equation of a straight line y=mx+B. The simplest of data fitting is, using the least squares, that is minimizing the sum of squared errors(or difference between the actual and predicted). The Least Squares is in fact a partial derivative of  (f(a,b) = a + bx )  [Ref: mathworld.wolfram.com]     [frac{delta(R^2)}{ delta a} = 2 sum_{i=1}^{n}[y_i-(a+bx_i) ]= 0]  [frac{delta(R^2)}{ delta b} = 2 sum_{i=1}^{n}[y_i-(a+bx_i) ]x_i = 0]  skipping to the final equation   [a = frac {hat{y}(sum_{i=1}^{n} x_i^2) - hat{x}sum_{i=1}^{n} x_i y_i} {sum_{i=1}^{n}x_i^2 - n hat{x}^2}] [b = frac {(sum_{i=1}^{n} x_i y_i) - n hat{x} hat{y}} {sum_{i=1}^{n}x_i^2 - n hat{x}^2}] Optimization techniques use slope to minimize the error or also called cost functions in Machine learning, A cost function is something you want to minimize. In…

## Campaign Efficiency – Retail Bank

A leading Retail Bank wanted to know the target customer profile, so, that the marketing campaign would be effective. The target is to disburse personal load to existing and new customers. Various key data were gathered. Using AI/ML, developed a model that would predict the probability(likelihood) that a customer would take a loan, given a set of customer profile data. Data Data collected during campaign can be used to Analyze and improve the efficiency. Key variables are Age, Job, A/C balance, Previous Loan, Marital status Analyses the data and clean the raw data Compare key variables with one another to establish any correlation Establish key variables that impact the outcome using feature engineering Build a ML model using Python using key variables Train using the key variables Test it against…

## HyperLedger Iroha – Setup and Configuration

It wasn't straight forward for me to setup Iroha, so, I decided to document here the steps , in case I forget ( I do within a week..) HyperLedger Iroha in docker environment using a Linux system (Debian or Ubuntu): Make sure docker and docker-compose are installed in the system. If not, sudo apt-get install docker docker-compose Clone the git repository to some local directory as follows: git clone https://github.com/hyperledger/iroha -b develop In the cloned repository, from the BASH shell, run the scripts/run-iroha-dev.sh. This will perform the following (will take some time to download the docker images from network): Docker instance for the build environment with a BASH shell mounted on /opt/iroha Docker instance for the redis server with server listening in the port number 6379 Docker instance for…

## Weka – Include / Exclude attributes

In Weka, there may be times when one wants to remove attributes from the data, through code. While I googled and did not find any, I set upon writing my own, Here is the snippet. For example the attributes are there in a file (attributes.txt), each line is an attribute, we read each attribute and find the corresponding index in the data(ARFF buffer bbcsport.arff). We then use Remove() class setAttributeIndicesArray() to set the array of indices to be either included or excluded by setting true or false in method remove.setInvertSelection(false). BufferedReader datafile = new BufferedReader(new FileReader("bbcsport.arff")); BufferedReader attrfile = new BufferedReader(new FileReader("attributes.txt")); Instances data = new Instances(datafile); List<Integer> myList = new ArrayList<Integer>(); String line; while ((line = attrfile.readLine()) != null) { for (n = 0; n < data.numAttributes(); n++) { if…

## Working with Raspberry Pi, 3g/ppp0, default gw, docker, conpot, some useful setups, etc

This is more of a bookmark for me, as I faced many setup related requirements that may be useful , the information is all over the web and one may not know it good they are. Hence this page in case I forget and am  lazy! First! Power up! Nothing happens without SD card!! Tried to install without display/keyboard/mouse by only using ssh, that's very tall requiment and ended up buying keyboard and mouse, used TV for display. Downlaod and install noobs or debian (no otherway !!) Now boot your Raspberry Pi 3 and ssh into it with user pi@x.x.x.x and the password raspberry Setup 3G USB modem, after a few hickups, got it going!  summarized here sudo apt-get update (important first step since also Wifi is setup..) lsusb and…