Compare the adjacent two elements and put the larger one the the
right then move to the next pair.
1.2 Implement
1 2 3 4 5 6 7 8 9 10 11 12
voidBubbleSort(int arr[], int length) { if(arr==NULL || length<=1) return;
for(int i=0; i<length-1; i++) // note: it should be length-1 { for(int j=0; j<length-1-i; j++) { if(arr[j]>arr[j+1]) swap(arr[j],arr[j+1]); // note: swap is also suitable for array } } }
1.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(n2)
O(1)
stable (no change in relative position for same
elements)
2. Selection Sort
(Improvement of Bubble)
2.1 Principle
Always try to find the smallest element from thoes have not be
sorted.
2.2 Implement
1 2 3 4 5 6 7 8 9 10 11 12 13 14
voidSelectionSort(int arr[], int length) { if(arr==NULL || length<=1) return;
for(int i=0; i<length-1;i++) // note: you do not to sort the last element { int min=i; // the index of the smallest element for(int j=i+1; j<length; j++) { if(arr[j]<arr[min]) min=j; } swap(arr[i], arr[min]); // put the min element to the head of array } }
2.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(n2)
O(1) (one more element for swapping)
unstable (the swap will let the same element
change their relative position)
3. Insertion Sort
3.1 Principle
[sorted, unsorted]. For each element in unsorted array, insert it to
sorted array.
3.2 Implement
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
voidInsertionSort(int arr[], int length) { if(arr==NULL || length<=1) return;
for(int i=1; i<length; i++) // note: you have to sort the first element { int key=arr[i]; int j=i-1; // compare this one with the elements before while(j>=0 && key<arr[j]) { arr[j+1]=arr[j]; j--; } arr[j+1]=key; // note: at this time: j=-1 or arr[j]<key, so it should be j+1 } }
3.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(n)/O(n2)/O(n2)
O(1) (one more element for swapping)
stable (no change in relative position for same
elements)
4. Shell's Sort
(Improvement of Insertion)
4.1 Principle
Divid the sequence to several subsequence. Sort this sequence
locally, then sort the whole sequence with insertion sort. eg. for
(0,1,2,3,4,5,6,7): (0,2,4,6) will be subsequence1 and (1,3,5,7) will be
subsequence2.
constint increment = 3; voidShellSort(int arr[], int length) { int step=length/increment+1; // you can choose increment=3 or anything else(<length)
while(step>=1) { // this part is same with insertion sort for(int i=step; i<length; i++) { int key=arr[i]; int j=i-step; while(j>=0 && key<arr[j]) { arr[j+step]=arr[j]; j-=step; } arr[j+step]=key; } step=step/increment; } }
4.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
The worst case is better than O(n2).
With certian increment, the worst case can be
O(n1.3)
O(1) (one more element for swapping)
unstable (the insertion sort will be performed
with different gaps)
5. Merge Sort
5.1 Principle
Based on Divide and Conquer.
Divid the sequence to two sub-sequence(new space). Sort them
seperatly, then merge two sub-sequence.
Note: There are two different ways of implementation: recursive and
unrecursive way. We only consider recursive way in C++.
// recursive way voidMerge(int arr[], int startIndex, int endIndex) { // first, we need to know where the two arrays are located. int leftMid=(startIndex+endIndex)/2; int rightMid=leftMid+1;
// we need to allocate new space for sorting int arrayLeft[leftMid-startIndex+1]; int arrayRight[endIndex-rightMid+1]; for(int i=startIndex; i<=leftMid; i++) arrayLeft[i-startIndex]=arr[i]; for(int i=rightMid; i<=endIndex; i++) arrayRight[i-rightMid]=arr[i];
// two pointer to merge int index_L=0, index_R=0; // merge to the original arr--to save space for(int i=startIndex; i<=endIndex; i++) { if(index_L != (leftMid-startIndex+1) && index_R != (endIndex-rightMid+1)) { if(arrayLeft[index_L]<arrayRight[index_R]) arr[i]=arrayLeft[index_L++]; else arr[i]=arrayRight[index_R++]; } // if left array is done elseif(index_L == (leftMid-startIndex+1)) arr[i]=arrayRight[index_R++]; // if right array is done else arr[i]=arrayLeft[index_L++]; } } voidMergeSort(int arr[], int startIndex, int endIndex)// since it is a recursive way, we need to save both indices { if(endIndex>startIndex) { MergeSort(arr, startIndex, (startIndex+endIndex)/2); // sort left part MergeSort(arr, (startIndex+endIndex)/2+1, endIndex); Merge(arr, startIndex, endIndex); // note: we need to sort in [startIndex, endIndex] } }
5.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(nlogn) with improvement, the best can be
O(n)
O(n)
stable the stability is ensured in Merge
6. QuickSort
(Important)
6.1 Principle
Based on Divide and conquer. As an improvement of BubbleSort.
It is the fastest sorting algorithm in average.
Select a pivot, put smaller (than pivot) elements in pivot's left and
larger elements in pivot's right. Do it recursively.
voidQuickSort(int arr[], int startIndex, int endIndex) { if(endIndex>startIndex) { int i=startIndex, j=endIndex; // select pivot int pivot=arr[startIndex]; // seperate the sequence while(i<j) { while(i<j && arr[j]>=pivot) j--; // if you select the most left element as pivot, you should go from right-->so that when i==j, the element will be smaller than pivot. if(i<j) arr[i++]=arr[j]; // put the smaller one to left while(i<j && arr[i]<=pivot) i++; if(i<j) arr[j--]=arr[i]; // put the larger one to right } // put the pivot, i==j arr[i]=pivot; // the original arr[i] is in arr[i+1/i-1]
// recursive QuickSort(arr, startIndex, i-1); // left side QuickSort(arr, i+1, endIndex); // right side } }
6.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(nlogn)/O(nlogn)/O(n2)
O(logn) (on average, the space complexity it the
depth of recursion; for the worst case, it can be O(n))
unstable (the swap of pivot makes it
unstable)
7. Radix/Bucket Sort
7.1 Principle
Sort input array using counting sort according to the i’th (from the
least significant digit to the most significant) digit.
// helper function, return the max number of digit intmaxBit(int arr[], int length) { int d=1; int p=10; for(int i=0; i<length; i++) { while(arr[i]>=pow(p,d)) d++; } return d; } voidRadixSort(int arr[], int length) { int d=maxBit(arr, length); // the max number of digit int temp[length]; int buckets[10]; // there are 10 buckets (0-9) int base=1; int remainder=0; for(int i=1; i<=d; i++) // you need to sort b times { for(int j=0; j<10; j++) buckets[j] = 0; // clear buckets // put arr[] into buckets for(int j=0; j<length; j++) { remainder=(arr[j]/base)%10; buckets[remainder]++; } // count the total number of elements that are stored in buckets for(int j=1; j<length; j++) buckets[j]+=buckets[j-1]; // you should go from back to front--make sure this algorithm is stable for(int j=length-1; j>=0; j--) { remainder=(arr[j]/base)%10; temp[buckets[remainder]-1]=arr[j]; buckets[remainder]--; } // put temp back to arr for(int j=0; j<length; j++) arr[j]=temp[j]; base=base*10; } }
7.2 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(nd) (d is the number of digits)
O(n+d)
stable
8. Heap Sort
8.1 Principle
Heap Sort is designed for heap. Heap is a complete binary tree and
its childen nodes should be always larger/smaller than parent nodes.
#### 8.2 Implement ```c++ // create maxheap void MaxHeapify(int arr[], int startIndex, int endIndex) { int parentNode=startIndex; int childNode=2*startIndex+1; while(childNode<=endIndex) { // compare the two child nodes, and choose the larger one if (childNode+1<=endIndex && arr[childNode]<arr[childNode+1]) childNode++; if (arr[parentNode]<arr[childNode]) { swap(arr[parentNode], arr[childNode]); parentNode=childNode; childNode=parentNode*2+1; } else return; } } void HeapSort(int arr[], int length) { // create the heap--start from the last parent node for(int i=length/2-1; i>=0; i--) MaxHeapify(arr, i, length-1);
for(int i=length-1; i>=0; i--) { swap(arr[0], arr[i]); // put the largest element to the end of array MaxHeapify(arr, 0, i-1); // rebalance the tree } }
8.3 Performance
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
O(n+nlogn) (n for creating the heap; rebalance
the tree(logn) for n times)
O(1) (one more element for swapping)
unstable (eg. 64, 27, 16, 27. After 64, the
second 27 will be ouput.)
Summary on Performance
Algorithm
Time Complexity (Best/Worst/Average)
Space Complexity
Stability
Bubble
O(n2)
O(1)
stable (no change in relative position for same
elements)
Selection
O(n2)
O(1) (one more element for swapping)
unstable (the swap will let the same element
change their relative position)
Insertion
O(n)/O(n2)/O(n2)
O(1) (one more element for swapping)
stable (no change in relative position for same
elements)
Shell
The worst case is better than O(n2).
With certian increment, the worst case can be
O(n1.3)
O(1) (one more element for swapping)
unstable (the insertion sort will be performed
with different gaps)
Merge
O(nlogn) with improvement, the best can be
O(n)
O(n)
stable the stability is ensured in Merge
Quick
O(nlogn)/O(nlogn)/O(n2)
O(logn) (on average, the space complexity it the
depth of recursion; for the worst case, it can be O(n))
unstable (the swap of pivot makes it
unstable)
Bucket
O(nd) (d is the number of digits)
O(n+d)
stable
Heap
O(n+nlogn) (n for creating the heap; rebalance
the tree(logn) for n times)
O(1) (one more element for swapping)
unstable (eg. 64, 27, 16, 27. After 64, the
second 27 will be ouput.)
Performance in test
Bucket Sort is the fastest sorting algorithm 👑 (it is also
stable)!
// Note: it seems the cache will affact the performance, so load the cache first. for(int i=0; i<10; i++) { shuffle_vector(input); Bucket_Sort(input); sorted=input; // get sorted list }
} voidMerge_Sort(vector<int>& vec, int start_index, int end_index) { int length=end_index-start_index+1; if(length==0) return; if(end_index>start_index) { Merge_Sort(vec, start_index, (start_index+end_index)/2); Merge_Sort(vec, (start_index+end_index)/2+1, end_index); Merge(vec, start_index, end_index); } } voidQuick_Sort(vector<int>& vec, int start_index, int end_index) { if(start_index>=end_index) return; // >= int i = start_index, j = end_index+1, pivot = vec[start_index];; while(true) { // the key point is '++i' and '--j' while(vec[++i]<pivot && i!=end_index); // find the first greater element while(vec[--j]>pivot && j!=start_index); // find the first smaller element if(i>=j) break; swap(vec[i], vec[j]); } swap(vec[start_index], vec[j]); Quick_Sort(vec, start_index, j-1); Quick_Sort(vec, j+1, end_index); } voidMax_Heapify(vector<int>& vec, int start_index, int end_index) { int parent_index = start_index, child_index = start_index*2+1; while(child_index <= end_index) { if(child_index+1<=end_index && vec[child_index]<vec[child_index+1]) child_index++; // find the max child element if(vec[parent_index]<vec[child_index]) { swap(vec[parent_index], vec[child_index]); parent_index = child_index; child_index = parent_index*2+1; } elsereturn; // since we go from bottom, there is no need to check the rest } } voidHeap_Sort(vector<int>& vec) { int length=vec.size(); if(length==0) return; // construct max_heap for(int i=length/2-1; i>=0; i--) Max_Heapify(vec, i, length-1);