[{"data":1,"prerenderedAt":970},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fplaylists\u002Fneural-networks\u002Fsigmoide-consertada-hinge-perdeu-margem":3,"post-en-neural-networks-sigmoide-consertada-hinge-perdeu-margem":4},"\u002Fplaylists\u002Fneural-networks\u002Fsigmoide-consertada-hinge-perdeu-margem",{"id":5,"title":6,"body":7,"cover":954,"date":955,"description":956,"extension":957,"meta":958,"navigation":959,"order":191,"path":960,"playlist":961,"seo":962,"status":963,"stem":964,"tags":965,"__hash__":969},"posts\u002Fen\u002Fplaylists\u002Fneural-networks\u002Fsigmoide-consertada-hinge-perdeu-margem.md","The Sigmoid Got Fixed, the Hinge Lost Its Margin",{"type":8,"value":9,"toc":943},"minimark",[10,25,30,83,118,139,150,201,247,259,267,295,302,322,357,373,377,419,422,456,526,537,547,657,675,679,686,698,709,713,765,769,775,795,837,847,861,901,908,925,936,939],[11,12,13,14,19,20,24],"p",{},"Lecture 4a, and it picks up exactly where ",[15,16,18],"a",{"href":17},"\u002Fen\u002Fplaylists\u002Fneural-networks\u002Ffuncoes-de-custo","last post"," left off. Same dataset, same pluggable ",[21,22,23],"code",{},"CostFunction"," framework, but the professor went back and touched two of the implementations. Worth checking line by line what changed, because one change fixes a real bug, and the other, without meaning to, creates a new one.",[26,27,29],"h2",{"id":28},"the-scaffolding-barely-changed-just-a-cosmetic-detail","The scaffolding barely changed, just a cosmetic detail",[31,32,37],"pre",{"className":33,"code":34,"language":35,"meta":36,"style":36},"language-python shiki shiki-themes github-light github-dark","def createDataset(n=20):\n  X = np.random.rand(n,2)\n  coefs = np.array([1, 1])\n  intercept = 1\n  labels = X @ coefs - intercept\n  y = np.sign(labels)\n  return X, y\n","python","",[21,38,39,47,53,59,65,71,77],{"__ignoreMap":36},[40,41,44],"span",{"class":42,"line":43},"line",1,[40,45,46],{},"def createDataset(n=20):\n",[40,48,50],{"class":42,"line":49},2,[40,51,52],{},"  X = np.random.rand(n,2)\n",[40,54,56],{"class":42,"line":55},3,[40,57,58],{},"  coefs = np.array([1, 1])\n",[40,60,62],{"class":42,"line":61},4,[40,63,64],{},"  intercept = 1\n",[40,66,68],{"class":42,"line":67},5,[40,69,70],{},"  labels = X @ coefs - intercept\n",[40,72,74],{"class":42,"line":73},6,[40,75,76],{},"  y = np.sign(labels)\n",[40,78,80],{"class":42,"line":79},7,[40,81,82],{},"  return X, y\n",[11,84,85,86,89,90,93,94,97,98,101,102,105,106,109,110,113,114,117],{},"The only real difference from ",[15,87,88],{"href":17},"last post's dataset",": ",[21,91,92],{},"y = np.sign(labels)"," instead of ",[21,95,96],{},"np.array(labels>0, dtype=int)*2-1",". Two different computations, the exact same result (",[21,99,100],{},"-1"," or ",[21,103,104],{},"+1","), just a more direct way to write the same thing. ",[21,107,108],{},"TrainingAlgorithm",", ",[21,111,112],{},"SGD",", and ",[21,115,116],{},"WidrowHoff"," came over unchanged, not a comma different, so I won't repeat them here.",[119,120,121],"blockquote",{},[11,122,123,130,131,134,135,138],{},[124,125,126,127,129],"strong",{},"Output (",[21,128,116],{},"):"," accuracy ",[21,132,133],{},"1.0",", weights ",[21,136,137],{},"[-1.98, 1.20, 2.36]",".",[26,140,142,145,146,149],{"id":141},"smoothedsurrogate-same-essence-get_cost-even-more-decorative",[21,143,144],{},"SmoothedSurrogate",": same essence, ",[21,147,148],{},"get_cost"," even more decorative",[31,151,153],{"className":33,"code":152,"language":35,"meta":36,"style":36},"class SmoothedSurrogate(CostFunction):\n  @staticmethod\n  def get_cost(y, y_pred):\n    y_pred = (y_pred >= 0).astype(int)\n    return np.sum(np.maximum(np.zeros(y.shape), -y * y_pred))\n  @staticmethod\n  def get_gradient(X, y, y_pred):\n    y_pred = np.sign(y_pred)\n    return X.T @ (y - y_pred)\n",[21,154,155,160,165,170,175,180,184,189,195],{"__ignoreMap":36},[40,156,157],{"class":42,"line":43},[40,158,159],{},"class SmoothedSurrogate(CostFunction):\n",[40,161,162],{"class":42,"line":49},[40,163,164],{},"  @staticmethod\n",[40,166,167],{"class":42,"line":55},[40,168,169],{},"  def get_cost(y, y_pred):\n",[40,171,172],{"class":42,"line":61},[40,173,174],{},"    y_pred = (y_pred >= 0).astype(int)\n",[40,176,177],{"class":42,"line":67},[40,178,179],{},"    return np.sum(np.maximum(np.zeros(y.shape), -y * y_pred))\n",[40,181,182],{"class":42,"line":73},[40,183,164],{},[40,185,186],{"class":42,"line":79},[40,187,188],{},"  def get_gradient(X, y, y_pred):\n",[40,190,192],{"class":42,"line":191},8,[40,193,194],{},"    y_pred = np.sign(y_pred)\n",[40,196,198],{"class":42,"line":197},9,[40,199,200],{},"    return X.T @ (y - y_pred)\n",[11,202,203,206,207,210,211,214,215,217,218,221,222,225,226,229,230,225,232,234,235,109,238,240,241,243,244,246],{},[21,204,205],{},"get_gradient"," stays identical in spirit to ",[15,208,209],{"href":17},"the perceptron criterion I already covered",": measures error after ",[21,212,213],{},"sign()",". ",[21,216,148],{}," changed (",[21,219,220],{},"y_pred >= 0"," becomes ",[21,223,224],{},"0","\u002F",[21,227,228],{},"1",", no longer ",[21,231,100],{},[21,233,104],{},"), but ",[15,236,237],{"href":17},"as I already noted",[21,239,148],{}," never enters training, only ",[21,242,205],{}," gets called by ",[21,245,112],{},". It gets even more crooked than before, but it still doesn't matter for anything.",[119,248,249],{},[11,250,251,130,254,134,256,138],{},[124,252,253],{},"Output:",[21,255,133],{},[21,257,258],{},"[-0.51, 0.56, 0.47]",[26,260,262,263,266],{"id":261},"the-real-fix-expit-finally-matches-the-label","The real fix: ",[21,264,265],{},"expit"," finally matches the label",[31,268,270],{"className":33,"code":269,"language":35,"meta":36,"style":36},"class LogLikehood(CostFunction):\n  @staticmethod\n  def get_gradient(X, y, y_pred):\n    y = (y>0).astype(int)\n    return X.T @ (y - expit(y_pred))\n",[21,271,272,277,281,285,290],{"__ignoreMap":36},[40,273,274],{"class":42,"line":43},[40,275,276],{},"class LogLikehood(CostFunction):\n",[40,278,279],{"class":42,"line":49},[40,280,164],{},[40,282,283],{"class":42,"line":55},[40,284,188],{},[40,286,287],{"class":42,"line":61},[40,288,289],{},"    y = (y>0).astype(int)\n",[40,291,292],{"class":42,"line":67},[40,293,294],{},"    return X.T @ (y - expit(y_pred))\n",[11,296,297,298,301],{},"Compare with ",[15,299,300],{"href":17},"the broken version I investigated last post",":",[31,303,305],{"className":33,"code":304,"language":35,"meta":36,"style":36},"# aula03b, the original bug\ndef get_gradient(X, y, y_pred):\n  return X.T @ (y - expit(y_pred))\n",[21,306,307,312,317],{"__ignoreMap":36},[40,308,309],{"class":42,"line":43},[40,310,311],{},"# aula03b, the original bug\n",[40,313,314],{"class":42,"line":49},[40,315,316],{},"def get_gradient(X, y, y_pred):\n",[40,318,319],{"class":42,"line":55},[40,320,321],{},"  return X.T @ (y - expit(y_pred))\n",[11,323,324,325,328,329,225,331,333,334,225,336,338,339,342,343,346,347,350,351,353,354,356],{},"The difference is a single line: ",[21,326,327],{},"y = (y>0).astype(int)",". That converts the ",[21,330,100],{},[21,332,104],{}," label to ",[21,335,224],{},[21,337,228],{}," ",[124,340,341],{},"before"," subtracting from ",[21,344,345],{},"expit(y_pred)",", which only ever returns values in ",[21,348,349],{},"(0,1)",". That's exactly the fix I predicted last post: line up the label's range with the sigmoid's range, instead of comparing ",[21,352,100],{}," (impossible for ",[21,355,265],{}," to reach) against a number that never goes negative.",[119,358,359],{},[11,360,361,130,363,134,365,368,369,372],{},[124,362,253],{},[124,364,133],{},[21,366,367],{},"[-5.44, 3.33, 5.82]",". Compared with the ",[21,370,371],{},"0.65"," I found before, the difference speaks for itself.",[26,374,376],{"id":375},"the-fixed-hinge-loss-that-actually-turned-into-something-else","The \"fixed\" hinge loss that actually turned into something else",[31,378,380],{"className":33,"code":379,"language":35,"meta":36,"style":36},"class HingeLoss(CostFunction):\n  @staticmethod\n  def get_gradient(X, y, y_pred):\n    y_pred = np.sign(y_pred)\n    marginal_errors = (y * y_pred) \u003C 1\n    marginal_ys = np.copy(y)\n    marginal_ys[~marginal_errors] = 0\n    return X.T @ marginal_ys\n",[21,381,382,387,391,395,399,404,409,414],{"__ignoreMap":36},[40,383,384],{"class":42,"line":43},[40,385,386],{},"class HingeLoss(CostFunction):\n",[40,388,389],{"class":42,"line":49},[40,390,164],{},[40,392,393],{"class":42,"line":55},[40,394,188],{},[40,396,397],{"class":42,"line":61},[40,398,194],{},[40,400,401],{"class":42,"line":67},[40,402,403],{},"    marginal_errors = (y * y_pred) \u003C 1\n",[40,405,406],{"class":42,"line":73},[40,407,408],{},"    marginal_ys = np.copy(y)\n",[40,410,411],{"class":42,"line":79},[40,412,413],{},"    marginal_ys[~marginal_errors] = 0\n",[40,415,416],{"class":42,"line":191},[40,417,418],{},"    return X.T @ marginal_ys\n",[11,420,421],{},"Again, compare with the earlier version:",[31,423,425],{"className":33,"code":424,"language":35,"meta":36,"style":36},"# aula03b\ndef get_gradient(X, y, y_pred):\n  marginal_errors = (y * y_pred) \u003C 1\n  marginal_ys = np.copy(y)\n  marginal_ys[~marginal_errors] = 0\n  return X.T @ marginal_ys\n",[21,426,427,432,436,441,446,451],{"__ignoreMap":36},[40,428,429],{"class":42,"line":43},[40,430,431],{},"# aula03b\n",[40,433,434],{"class":42,"line":49},[40,435,316],{},[40,437,438],{"class":42,"line":55},[40,439,440],{},"  marginal_errors = (y * y_pred) \u003C 1\n",[40,442,443],{"class":42,"line":61},[40,444,445],{},"  marginal_ys = np.copy(y)\n",[40,447,448],{"class":42,"line":67},[40,449,450],{},"  marginal_ys[~marginal_errors] = 0\n",[40,452,453],{"class":42,"line":73},[40,454,455],{},"  return X.T @ marginal_ys\n",[11,457,458,459,462,463,470,471,474,475,478,479,482,483,486,487,490,491,494,495,498,499,502,503,506,507,109,509,511,512,514,515,517,518,521,522,525],{},"The only change is the line ",[21,460,461],{},"y_pred = np.sign(y_pred)",", added right at the top. It looks harmless, ",[15,464,466,467,469],{"href":465},"\u002Fen\u002Fplaylists\u002Fneural-networks\u002Fmcculloch-pitts-perceptron","the same ",[21,468,213],{}," that's shown up throughout this playlist",", but it changes the entire meaning of ",[21,472,473],{},"marginal_errors",". Before, ",[21,476,477],{},"y * y_pred"," used the ",[124,480,481],{},"continuous"," pre-activation, so ",[21,484,485],{},"y * ŷ \u003C 1"," covered both a misclassified point (",[21,488,489],{},"y * ŷ \u003C 0",") and a correctly classified one ",[124,492,493],{},"with no room to spare"," (",[21,496,497],{},"0 ≤ y * ŷ \u003C 1","), those latter ones are the \"support vectors\" ",[15,500,501],{"href":17},"I explained last post",": hinge loss keeps pushing the boundary until they get some breathing room. Now, with ",[21,504,505],{},"y_pred"," already passed through ",[21,508,213],{},[21,510,477],{}," can only be ",[21,513,100],{}," (wrong) or ",[21,516,104],{}," (right), never anything in between. ",[21,519,520],{},"\u003C 1"," becomes a synonym for \"just wrong\", which is exactly ",[15,523,524],{"href":17},"the perceptron criterion"," again, just written with more lines.",[119,527,528],{},[11,529,530,130,532,134,534,138],{},[124,531,253],{},[21,533,133],{},[21,535,536],{},"[-0.58, 0.53, 0.59]",[11,538,539,540,543,544,546],{},"To confirm this isn't me nitpicking one line of code, I built a concrete example: took an already-trained boundary (",[21,541,542],{},"w = [-2.5, 3.0, 2.75]",", which separates the 20 points with ",[21,545,133],{}," accuracy) and checked, point by point, who each rule considers \"marginal\" (i.e., who still contributes to the gradient).",[548,549,550,583],"table",{},[551,552,553],"thead",{},[554,555,556,561,567,570,577,580],"tr",{},[557,558,560],"th",{"align":559},"center","Point",[557,562,564],{"align":563},"left",[21,565,566],{},"x0, x1",[557,568,569],{"align":559},"class",[557,571,573,576],{"align":572},"right",[21,574,575],{},"y·ŷ"," (continuous margin)",[557,578,579],{"align":559},"Old rule (aula03b)",[557,581,582],{"align":559},"New rule (aula04a)",[584,585,586,612,634],"tbody",{},[554,587,588,592,597,601,606,609],{},[589,590,591],"td",{"align":559},"6",[589,593,594],{"align":563},[21,595,596],{},"(0.091, 0.810)",[589,598,599],{"align":559},[21,600,100],{},[589,602,603],{"align":572},[21,604,605],{},"0.001",[589,607,608],{"align":559},"marginal (adjusted)",[589,610,611],{"align":559},"ignored",[554,613,614,616,621,625,630,632],{},[589,615,224],{"align":559},[589,617,618],{"align":563},[21,619,620],{},"(0.452, 0.560)",[589,622,623],{"align":559},[21,624,104],{},[589,626,627],{"align":572},[21,628,629],{},"0.397",[589,631,608],{"align":559},[589,633,611],{"align":559},[554,635,636,639,644,648,653,655],{},[589,637,638],{"align":559},"8",[589,640,641],{"align":563},[21,642,643],{},"(0.982, 0.965)",[589,645,646],{"align":559},[21,647,104],{},[589,649,650],{"align":572},[21,651,652],{},"3.100",[589,654,611],{"align":559},[589,656,611],{"align":559},[11,658,659,660,663,664,666,667,670,671,674],{},"Point 6 is the most telling: its pre-activation is ",[21,661,662],{},"-0.001",", practically sitting on the boundary, exactly the kind of point that should keep \"bothering\" training. The old rule (continuous margin) recognizes that and keeps adjusting its weight. The new rule, having already applied ",[21,665,213],{},", only sees that it landed (by the barest margin) on the correct side, and gives up on it. In total, ",[124,668,669],{},"9 of the 20 points"," fall into that band (",[21,672,673],{},"0 ≤ y·ŷ \u003C 1",", correct but with no room to spare): the old rule keeps adjusting the weight on them, the new rule considers all 9 already settled.",[26,676,678],{"id":677},"the-four-boundaries-from-this-lecture-side-by-side","The four boundaries from this lecture, side by side",[11,680,681,682,685],{},"Same visual exercise ",[15,683,684],{"href":17},"as last post",", now with the real weights this fixed version found:",[687,688],"classification-boundary-comparison-chart",{":boundaries":689,":classes":690,":points":691,":x-max":692,":x-min":693,":y-max":692,":y-min":693,"negative-label":694,"positive-label":695,"x-label":696,"y-label":697},"[{\"w\":[1.20352648,2.36049089],\"bias\":-1.98404527,\"label\":\"Widrow-Hoff\"},{\"w\":[0.56431986,0.4707198],\"bias\":-0.50817736,\"label\":\"Perceptron\"},{\"w\":[3.32853637,5.8209982],\"bias\":-5.44082457,\"label\":\"Log-likelihood (fixed)\"},{\"w\":[0.52695741,0.59010705],\"bias\":-0.57855574,\"label\":\"Hinge (aula04a)\"}]","[-1, 1, 1, 1, -1, 1, 1, -1, 1, -1, 1, -1, 1, -1, -1, 1, -1, -1, -1, -1]","[[0.206, 0.397], [0.815, 0.237], [0.763, 0.289], [0.375, 0.993], [0.514, 0.237], [0.62, 0.801], [0.942, 0.064], [0.61, 0.389], [0.234, 0.784], [0.527, 0.045], [0.945, 0.578], [0.366, 0.132], [0.906, 0.267], [0.192, 0.677], [0.137, 0.292], [0.789, 0.264], [0.017, 0.517], [0.663, 0.146], [0.617, 0.129], [0.01, 0.342]]","1.1","-0.1","class -1","class +1","x0","x1",[11,699,700,701,704,705,708],{},"Once again all four converge to pretty much the same line, on this easy dataset. The difference the analysis above uncovered isn't in ",[124,702,703],{},"where"," the final boundary lands, it's in ",[124,706,707],{},"which points keep getting adjusted"," throughout training, something this chart (just the final result) can't show on its own.",[26,710,712],{"id":711},"wrapping-up","Wrapping up",[548,714,715,725],{},[551,716,717],{},[554,718,719,722],{},[557,720,721],{"align":563},"What I already knew",[557,723,724],{"align":563},"What this lecture settled",[584,726,727,746,757],{},[554,728,729,739],{},[589,730,731,732,734,735,738],{"align":563},"I found the ",[21,733,265],{},"-versus-",[21,736,737],{},"±1","-label bug",[589,740,741,742,745],{"align":563},"The professor confirmed the diagnosis: converting the label to ",[21,743,744],{},"{0,1}"," fixes it, exactly as I predicted",[554,747,748,751],{},[589,749,750],{"align":563},"Hinge loss uses a continuous margin to decide who's a \"support vector\"",[589,752,753,754,756],{"align":563},"Applying ",[21,755,213],{}," before measuring the margin collapses that into \"right or wrong\", losing the notion of margin entirely",[554,758,759,762],{},[589,760,761],{"align":563},"Fixing a bug can, without meaning to, introduce another one",[589,763,764],{"align":563},"Worth checking each fix line by line, even when the final result (accuracy) looks good",[26,766,768],{"id":767},"practical-application","Practical application",[11,770,771,772,774],{},"I repeated the comparison, real hinge (continuous margin) vs. aula04a hinge (margin via ",[21,773,213],{},"), on two real datasets already used in this playlist.",[11,776,777,778,782,783,109,786,790,791,794],{},"On Iris (",[779,780,781],"em",{},"setosa"," vs. ",[779,784,785],{},"versicolor",[15,787,789],{"href":788},"\u002Fen\u002Fplaylists\u002Fneural-networks\u002Fperceptron-com-bias","again","), I ran both variants with the same ",[21,792,793],{},"learning_rate=0.01"," and the same seed:",[548,796,797,810],{},[551,798,799],{},[554,800,801,804,807],{},[557,802,803],{"align":563},"Version",[557,805,806],{"align":572},"Train accuracy",[557,808,809],{"align":572},"Test accuracy",[584,811,812,823],{},[554,813,814,817,819],{},[589,815,816],{"align":563},"Real hinge (continuous margin)",[589,818,133],{"align":572},[589,820,821],{"align":572},[124,822,133],{},[554,824,825,831,833],{},[589,826,827,828,830],{"align":563},"aula04a hinge (margin via ",[21,829,213],{},")",[589,832,133],{"align":572},[589,834,835],{"align":572},[124,836,133],{},[11,838,839,840,843,844,846],{},"A technical tie, and it makes sense: ",[15,841,842],{"href":788},"Iris has too generous a margin"," for any point to be left in the ",[21,845,673],{}," band once it converges, so the two rules end up agreeing on who's \"marginal\": nobody.",[11,848,849,850,109,857,860],{},"I switched to the two-moons dataset (",[15,851,853,856],{"href":852},"\u002Fen\u002Fplaylists\u002Fneural-networks\u002Fxor-o-limite-do-perceptron",[21,854,855],{},"make_moons",", already used in the XOR post",[21,858,859],{},"noise=0.25"," this time to guarantee real overlap between the classes, no perfect linear separation) and ran 20 different seeds for each version:",[548,862,863,875],{},[551,864,865],{},[554,866,867,869,872],{},[557,868,803],{"align":563},[557,870,871],{"align":572},"Test accuracy (mean of 20 seeds)",[557,873,874],{"align":572},"Weight vector norm (mean)",[584,876,877,889],{},[554,878,879,881,886],{},[589,880,816],{"align":563},[589,882,883],{"align":572},[124,884,885],{},"0.878",[589,887,888],{"align":572},"2.90",[554,890,891,895,898],{},[589,892,827,893,830],{"align":563},[21,894,213],{},[589,896,897],{"align":572},"0.842",[589,899,900],{"align":572},"0.67",[11,902,903,904,907],{},"With data where the linear boundary isn't perfect, the difference shows up: real hinge beats the aula04a version on test accuracy across the 20 seeds, and the weight norm is almost ",[124,905,906],{},"4 times bigger",", exactly because it keeps adjusting the boundary on points that land correctly but with no room to spare, pushing toward a more confident separation instead of stopping as soon as the sign already matches. That's the practical proof that a continuous margin isn't just a theoretical detail, it changes the outcome once the dataset isn't trivially separable.",[11,909,910,911,913,914,917,918,920,921,924],{},"Taking one specific seed out of those 20 (test accuracy ",[21,912,885],{}," for real hinge, ",[21,915,916],{},"0.789"," for the ",[21,919,213],{}," version, close to the average of both), plotting both boundaries over the ",[124,922,923],{},"90 test points",", the same ones already used in the ELM post:",[687,926],{":boundaries":927,":classes":928,":points":929,":x-max":930,":x-min":931,":y-max":932,":y-min":933,"negative-label":934,"positive-label":935,"x-label":696,"y-label":697},"[{\"w\":[0.897,-2.763],\"bias\":0.18,\"label\":\"Real hinge (continuous margin)\"},{\"w\":[0.066,-0.627],\"bias\":-0.06,\"label\":\"aula04a hinge (margin via sign())\"}]","[1, 1, 1, 1, 1, -1, 1, -1, 1, 1, -1, -1, 1, -1, 1, 1, -1, 1, -1, -1, -1, 1, -1, -1, 1, 1, -1, 1, -1, 1, 1, -1, -1, -1, -1, 1, -1, 1, 1, 1, 1, -1, -1, 1, -1, 1, 1, 1, 1, 1, 1, 1, 1, 1, -1, -1, -1, -1, 1, -1, 1, -1, 1, 1, -1, -1, 1, -1, -1, 1, 1, -1, -1, 1, 1, 1, -1, -1, -1, 1, -1, 1, 1, -1, 1, 1, 1, 1, 1, 1]","[[0.459, -0.307], [2.533, 0.563], [0.775, -0.587], [0.313, 0.02], [1.919, -0.722], [-0.467, 0.153], [1.938, 0.643], [-0.059, 0.977], [1.001, -0.314], [1.285, -0.493], [-0.569, 0.954], [-0.516, 0.844], [0.062, 0.484], [0.626, 1.302], [1.613, 0.5], [1.921, 0.04], [-0.883, 0.769], [1.574, 0.382], [1.2, -0.04], [-1.196, 0.011], [0.07, 1.035], [1.268, -0.456], [0.976, 0.071], [-0.776, 0.618], [1.839, -0.179], [1.958, -0.004], [1.424, 0.352], [0.508, -0.064], [0.089, 1.376], [1.53, -0.02], [0.725, -0.242], [0.735, 0.453], [0.764, 0.067], [-1.03, 0.902], [0.265, 1.636], [0.992, -0.205], [-0.404, 0.661], [0.519, -0.287], [1.167, -0.596], [0.576, -0.121], [1.29, -0.046], [-0.235, 1.188], [-1.088, 0.061], [1.0, -0.268], [-0.046, 1.319], [0.093, 0.118], [0.234, -0.23], [0.364, -0.531], [1.369, -0.069], [1.319, -0.777], [0.356, -0.179], [0.542, -1.002], [1.024, -0.46], [1.414, 0.592], [-0.73, 0.946], [0.149, 0.648], [-0.741, -0.41], [0.029, 1.456], [1.178, -0.145], [0.529, 0.794], [2.187, 0.293], [0.955, 0.648], [0.321, -0.131], [1.787, 0.148], [0.508, 0.51], [1.277, 0.558], [1.535, -0.584], [0.924, 0.607], [-0.236, 0.773], [1.57, -0.272], [0.271, -0.032], [0.692, 0.917], [0.399, 0.695], [0.374, -0.947], [2.224, -0.191], [0.046, 0.335], [0.412, 0.782], [-0.265, 0.983], [0.897, 0.68], [0.494, 0.467], [0.625, 0.516], [2.185, 0.273], [0.382, 0.436], [-1.1, 0.543], [2.113, 0.488], [1.479, 0.104], [0.197, -0.404], [1.417, -0.792], [1.153, -0.648], [1.418, -0.025]]","2.6","-1.3","1.7","-1.1","moon 0","moon 1",[11,937,938],{},"You can see the \"confidence\" difference between the two lines: real hinge (bigger weight) cuts more perpendicular to the overlap band between the moons, taking on more risk to separate better; the aula04a version, with a small weight, stays more \"lukewarm\", stopping adjustment as soon as clearly correct points fall out of the sum, even with no room to spare.",[940,941,942],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":36,"searchDepth":49,"depth":49,"links":944},[945,946,948,950,951,952,953],{"id":28,"depth":49,"text":29},{"id":141,"depth":49,"text":947},"SmoothedSurrogate: same essence, get_cost even more decorative",{"id":261,"depth":49,"text":949},"The real fix: expit finally matches the label",{"id":375,"depth":49,"text":376},{"id":677,"depth":49,"text":678},{"id":711,"depth":49,"text":712},{"id":767,"depth":49,"text":768},null,"2026-08-27","Lecture 4a: the professor comes back and fixes the exact sigmoid-versus-±1-label mismatch I found last lecture. But checking it line by line, the hinge loss fix hid a new problem: it accidentally turned back into the perceptron criterion.","md",{},true,"\u002Fen\u002Fplaylists\u002Fneural-networks\u002Fsigmoide-consertada-hinge-perdeu-margem","neural-networks",{"title":6,"description":956},"published","en\u002Fplaylists\u002Fneural-networks\u002Fsigmoide-consertada-hinge-perdeu-margem",[966,967,968],"cost-functions","hinge-loss","log-likelihood","jH-eLyy0E4XPVip0H4fQWj5xTrCy7xEPhCGs0V5C2Ak",1788965100267]