FineBI Quick Start

What is FinBI

FineBI是商业自主分析软件，类似大众熟悉的Tableau和PowerBI，相比于使用Excel的优势在于

1. 轻量，速度快（部署在远端服务器，本地的操作全部在网页完成）
2. 支持数据库同步（链接数据库，数据下载和Query查询都可以实现，而且更快更简单）
3. 丰富的对图/表的支持，和Excel透视表功能类似，同时已经构建了大量常用功能

Induction Reasoning

Induction reasoning aims at developing a theory. It moves from specific observations to general conclusions. The quality of the conclusions obtained via induction depends on the quality of observations and analysis.

eg.

Observations:

Howard is a men. Howard is mortal.

Conclusion:

All men are mortal.

Note

Mathematical Induction should not be confused with induction reasoning used in philosophy. This mathematical methods examines infinitely many cases to prove a general statement by a finite chain of deductive reasoning.

Deduction Reasoning

Deduction reasoning aims at testing an existing theory. It formulate hypothesis and accept/reject it with data and facts. Deduction is idea-first followed by observations to draw a conclusion. Deduction's conclusions are solid.

eg.

Theory: All men are mortal.

Observation: Howard is a men.

Conclusion: Howard is mortal.

1. Install Unity (2022 LTS) and VSCode.
2. Setup .Net Environment
   1. install Mono from stable channel
   2. install SDK
3. Change Unity Settings
   1. Create a unity project
   2. Set VSCode as default editor. Settings->External Tools->External Script Editor. Set it to Visual Studio Code.
4. Asset->Open C# Project. To open project in VSCode.
5. VSCode Pulgins:
   1. C#: setup C# environment
   2. Unity Code Snippets: auto code completion
   3. Unity Snippets
   4. Unity Tools
   5. C# XML Documentation Comments: generate comments and documents
6. VSCode Setting
   1. Turn off omnisharp.usemodernnet
   2. Set omnisharp.monopath
   3. (optional) Turn off telemetry (it sends analytics to Microsoft)
7. Restart VSCode.

python里and or的规则：强制转化为bool类型，表达式的值是从左到右第一个能够确定表达式的变量的值

1
2
3
4
5
6
7
8
9
10

>>> 0 or 1
1
>>> 0 or 1 or 2
1
>>> 0 or 1 and 2
2
>>> 0 or 1 or 3 and 2
1
>>> 0 or 1 or 3 and 0
1

Counter is a pythonic way to handle counting repeating elements in a list.

1
2

from collections import Counter
Counter([1,2,3,3,4,3,2]) # actually this is a dictionary

1. if it is about aggregate data -- copy and paste (minus)

2. if it is about detail data -- try datacompy in python

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   import datacompy # pip install datacompy import pandas as pd import sys files = [] try: files = [sys.argv[1], sys.argv[2]] sheet = int(sys.argv[3]) except: raise Exception('No Input') df1 = pd.read_excel(files[0], sheet_name=sheet) df2 = pd.read_excel(files[1], sheet_name=sheet) join_columns = ['xxx'] # key to join compare = datacompy.Compare(df1, df2, join_columns=join_columns, rel_tol=0.01, # difference between two float <= 0.01 df1_name='data 1', df2_name='data 2' ) print(compare.matches()) # return bool print(compare.report()) # return detailed report

   what it does is joining two tables and comparing each column and each row.

HEAD is a symbolic ref pointing the branch which is also a ref pointing the latest commit oid. like this

1
2

$ cat .git/HEAD 
ref: refs/heads/main

In this case, if u make a new commit, git will not actually change the HEAD but change the branch pointed by HEAD. Bad thing happens if u checkout a commit by its oid. Checking a commit by its oid means HEAD will be not be a symbolic ref and HEAD will point at a commit directly. like this

1
2
3

$ git checkout e53fae9913340f71ade680e5b1c477892a2abd90
$ cat .git/HEAD 
e53fae9913340f71ade680e5b1c477892a2abd90

e53fae9913340f71ade680e5b1c477892a2abd90 is a commit id.

U will get a warning when u try git checkout <commit oid>

1
2
3
4
5

Note: switching to 'e53fae9913340f71ade680e5b1c477892a2abd90'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

Now if u do some changes and commits, HEAD will record everything. But if u switch to a branch, u will lose the changes and commits because there is no branch tracking ur modification (these commits are recorded by git but u can only track them via their oid). U should avoid this situation (create a new branch instead), or tag the commits before leaving.

数仓分层:

ODS(Operational Data Store): 原始数据、和业务系统保持一致 (ODS layer stores real-time layer while Data Warehouse stores historical data.)

DWD(Data Warehouse Detail): 以业务需求为驱动，将原始数据清洗整合为事实表

DWS(Data Warehouse Summary): 以分析的主题对象为建模驱动，构建宽表

ADS(Application Data Service)/DM(Data Mart): 最终根据需求建立Table/Dashboard

MILP -- Mixed Integer Linear Programming

Highly recommended: CVXPY

CVXPY is an interface which is compatible with multiple solvers like CBC, CPLEX, GUROBI, etc.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import numpy as np
import cvxpy as cp
n = 10
np.random.seed(3)
a = np.random.randint(1, 10, size=n)
b = np.random.randint(1, 10, size=n)

# Variables
x = cp.Variable(shape=n, boolean=True)
# Objective
objective = cp.Minimize(x @ a)
# Constraints
constraints = []
constraints.append(x @ b >= 10)

# Build & Solve Problem
problem = cp.Problem(objective, constraints)
problem.solve(solver=cp.CPLEX, verbose=0)

# Print Result
print(problem.status)
print("x: ", x.value)
print("Optimal value: ", problem.value)

Modify git commit message

1. switch git commit editor from NANO to VIM: git config --global core.editor vim
2. Edit the most recent commit message:
   1. git commit --amend or git commit --amend -m "New commit message."
   2. before changing commit message, you can also add other changes via git add <files>
3. Edit historical commit message:
   1. git rebase -i HEAD~N (modify \(N^{th}\) latest commit message) or git rebase -i <commit id> (you can only modify commit message after <commit id>) or git rebase -i --root (modify from root commit).
   2. change pick to reword => save and close editor :wq => for each chosen commit, a new text editor will open.
   3. force push it git push --force <remote name> <branch name>
   4. note: You should avoid amending a commit that is already pushed.