ABOUT THIS EPISODE
Episode 90 · 1 year ago
SHARE THIS EPISODE
Episode 90 · 1 year ago
Hashing It Out #90- Core Scientific-Ian Ferreira
ABOUT THIS EPISODE
I las up so evalent. Let's talk aboutit, l, listen avalanche. The snow comes down real fast FERC games momentum, butI'm not talking about the natural disaster or it's not really. DisasterGus Hof, no one's Aron but anyway avalanche. What you've heard about it,not yer some more. It's an open source platform, envalanching disentralizedfinance application, rigt DEFA, that's what'! Your want developers who build on avalanche caneasily create powerful, reliable, secure applications and custom.Blockchan networks with complex rule sets or build an existing private forpublic subnat right. I think what you should do right now is stop what you'redoing, even if it's listening to this podcast stop pull over go to the gasation. If you need to goto a subway, there's a subway like everywhere, there's always a subwayaright, a there's, always a Croger Ju stop in a parking lot somewhere! OP. The dogs are adamant that you stop go to Avalab hovolahavs Tahorg, to learn more. All right, stop go toalvalabs thats, ABA labs, labs, NOTORG, ntiind work! Welcome to hashing it out APOASS forretalk to the ATTECH intevators, behind blocked in intrastructure anddecentralized networks. We dive into the weeds toget at Wy and how peoplebuild this technology, the problems they face along the way, I'm listeningto d learn from the best in the business. You can join EIR, reck Boebackash out O petty with Mycohost Dean Igenman.Today's episode, we're going to be talking with core scientific recentlyhad a potcass at them over on the bicin potcast with a different person from acompany, and we wanted to get a little more technical. So we brought overinfeera from chief, a chief product officer, of course, scientific in dothe standard thing and Kindo tell us what you do where you came from and howyou joing this space ey good morning. Yes, I'm Infrera, I'mchief product officer, core scientific vebeen, with coure scientificapproaching two years now, um before that Ih d, bounced around a couple ofother machine, leaning, startups and spent a decade at Microsoft, working inthe search team to have been around the algorithms and big data distributedsystem Spaye, my entire carermostly. But you do now like what kind ofbrought you a coure scientific, and what do you do there? So course sitific was interesting for acouple of reasons. One is, I definitely wanted to focus on AI, so that was oneof my criterian. The second was it's a very different approach. So a lot ofcompanies if you're go ind working an airol you're, going to start frombusiness problems downwards and kind of make at way to you know, let's say tensorflow kind of that. U Know tensor flow by George Layer up the stock and what was unique about corescientific is. We were starting from the bottom, so we were starting fromconcrete power. TDP chipsits, you know understanding into connectad really anopportunity to get the underthehood experience. If you wole the hardwareexperience, if you will of Ai and then work your way op, so that once you'vedone that you have a full picture of everything from okay. This is going touse this library. That's going to take advantage of this selicon feature.That's Gongto be accelerated by this hardware infrastructure and Blah BlahBlah. So it just gave me a really unique opegunity to work on theBOTTOMUP. That's really interesting, so you'relike because for TST, the don't know course scientific is, I guess you boast,yourselfs as a infrastructure of company providing a lot of resourcesfor people to do a myriad of things that ppraneed compute power, part ofthat being mining d, varioscrapal carces, as Wollas like machine learning and so and so forth, and ai H, hit's, interesting that you you take itfrom these. Are the resources that we have. These are the architectures thatmaybe fit to these different types of alborithms that are applied across theboard. What dos that be? Even that perspectivelike how do you? How do you approach a problem? And what is that kind o? Whathave you learned from that Yeahso? As you mentioned course,entific provides...
...hosting and infrastructure s one of thesoftware services for the two primary categories. One is block Jane and theother AI and the block chains side were no lower down the stacks. You know,somewhere between Detisen as e service and infrastructures of service, wherewe host mining gear, full customers and then onthe isside wee much higher up. This stack so we're pretty much pass Batfom as a service and we'll talk about that some morelater. But those are the kind of the two differences and then we have youknow some synogies between the two. If you start at the facilityes level nowthe common thing with crypto and an ai gear is hi, he tigh power, so ourfacilities are typically much higher rated than you would find in atraditional deta center and the other aspects is around controlling heat. I'm Instoa ofmaking sure you can deal with these machines that don't sit in normalstandard racks that you might be used to. So that's the infrastructure, tere Um,and then we do a couple of things around using algorithms around a Workloplacement. We do that Boz in Ain on Blockcan Um. So, as you can imagine, ifyou look at on the block Jane side, a very common work, clothe that you mightgant to figure out, what's the optimal coin to mine right. So how do you dothat? You have to figure out a bunch of algorithms and ingredients to make a decision what to mind, and wecan. We can talk a little bit more about how that works and the same thingon the AI side. You might want to run a large training job Um, and you mightwant to notice this better to run an Azure. Is it better to run on ourinfrastructure cores it better to run on aws, because again you have the sameequations, there's a cost and there's a compute capability and you have tofigure out what' the optimal for the customers need so that's kind of thesufferer ovelap that we have between the two verticals. That makes sense you kind of justanswered my or partially answered my next question, which was why why blockchain and a e it seemslike too, very abstract, different things? Why? Whynot focus on one? Why did you guys decide to do both of them? was therethat much overlap that it made sense or yeah? That's a great question. I thinkthe company started before c o Kamand Turner joined Athe Company wasprimarily focused on Blockchang an crostructure, and I think he saw theopportunity to expand into ai because of that similarity, Um and you knowwe've. You know. Ur Ai Business is much more Nacion than block chain, but we'vegrown pretty substantially and, as I mentioned, we've gone much higher upthe stack Um. You know the giving people this cloud experience to gainaccess to this inprastructure. This is on the block, chainge side, whith, moreof an is or Ordata centers a service where we managing the infrastructure.Full customers do do you? Are there any used caseswhere you use? Let's say the one vertical in the other, for example, isthere a use case for some of your block train or for some of your AiInfrastructuring, your blockching product, or are theycompletely orthogonal? It's a great question, absolutely so going back to the earlier. You knowworkloud placement optimization in the case of block Jane The workcloud couldbe, you know which crypte currency are you mining and we have something calleddeep mine, which is a ML based recommendation engine which would tellthe customers they can opten to use it if they wanted to it'll tell them heybase on a couple of signals which we can dig into later. We recommend that the most profitablecoin for you to mind right now is x, and then we could automatically usingour infrastructures, switch their entire freed over to that coin, and wedo that every thity minutes we calculate the profitability Um and enand those are based on Ai Models Um. I think the the most natural sit forblockchain into AI, so the reverse of that is around io t. So if you look atthe massive amounts of data that's being collected on the edge, it's ahigh volume, low value data and so being able to put than on an immutablestorage is critical for Hae. So you can retrain models, reproduce results and I think that's where, at least inmy mind, I see the overlap of...
...blockchain. PROVIDING UTILITY TO AI wasyer, saying that the data ingestion of a lot of these aitude devices goinginto some block chaines of store, storage, Mechodist, St Right, Ri,correct, correct 'cause, you Le immutable storage, but you need a lotof it. 'CAUSE IT might be. Temperature censors,and so one reading getting lost is not the end of the world, but you do want oMay, but you're going to have a ton of readings and from a ton of places o youneed to distributeit it Um. You know immutable storage for CENCORDATA AND IMMUTABILITY HOA scientists reproduce results, so you don't have toworry about theemperatures being jip out after the fact frexits advantage of that. I understandthe concept there, but a disadvantage of trying to pump a bunch of data intoa BLATATA Statebloa. If you look at m one of the current, I guess bottle nextof the th of network. It's how much state management currentlyexists and how much you have to kind of H. Get through in order to just becomea running full noad across the network and if you'd like to run something lite,an archive Toud, which ha handles all historical data, then it's it becomes severely limitingin terms of the resources required to run these things and yeah. That's justdealing with, like you know, smart contract DAA financial data, whereyou're really spending a lot of time, optimizing what you're putting there.You start doping about of stuff like Ote data, that's Gonta, get exacerbatedterribly I'd. Imagine something along the linesof like using a blocksion for yoummutability, but storing a good portion of that data elsewherein hashing it. So you Hav, like you'rerooting, you'rerooding your undibility, on something like a Blockcan, but I don't think youcan scale realiity data on Bonoplack, my pane yeah. No, it's Um and and againth sist of the Alment of bug, block jen and ai they're. Such ey used toexpression, suitcase words. They mean so many things. I would absolutelyagree that putting that on a cerium or or the betcin network is an overkill,they have much higher primitives and capabilities that won't be needed andUm. You know argibly having to generate consensus across this type of datashouldn't be such such mission, criticalds, alsoobmission, criticalcompared to other use guses, but there's two ways to solve that. One isto dial down the tolerance of consensus to use a simplified ledger. The otheris. I should mention to keep the transactional aspects and a distributedleger, but keep the storage elsewhere either way. As Long Isyo, you knowthe H,Guite anquote off te chain depoyments as long as you're able to reasonablycontrol the data and be able to account for the origins,I think there's still utility there. It makes hi think I wan I kind of divea little more into what you said like as s the underlike usecase. What is theissue with H M a? I guess this. This also do tailsthat something I wanted to talk about later. You're dealing with machinelearning and a data. The datathinkour injesting is incredibly important old mentality of carbage end garbageout. You can create models on on data all day long, but if, if what you'retrying to get out of it and the day to your feeding into it, don't aren'treconciled very well you're going to get ship models that don't give you anyPunh preecting power Um, and so you w what you're looking for at least trainto leverage this other technology is a way to have stronger confidence in the quality of the data.Theniger dressing is absoutel like universal too. The machinelearning Industryi is the oracles. I guess you can call these oracles asevere problem. Yeah I mean you nailed it and it's no different from fromhuman minds, you're a function of whatever data you exposed to so samething with neural networks. If you have data sense that are skewed or you haveclass and balance problems en your credictof models will produce thes sameproblems, and so it's important to have stable data sets thatve been curatedand publicly available a distributed. So you don't have to move large amountof data around the world and stable they. They need to be immutable so thatyou can reproduce results. If, if you have ch concernts that thedatasets might be drifting, then ever be really difficult to produce resultsand and be able to do hypoparameter cuning on your models so yeah I meanthat that's the use case I see for Blockchain, N, Ai Um and, like I said there, ere vers.Obviously we spoke about more...
...s. So add a quick question based onsomething you said previously in one of the first answers when I asked youwhich was related to you said that you guys every thirty minutes will checkwhich crypto currency is the most profitable toprovidable to Min y andthen switch to that one with me with. Maybe this is because I'm a bit naiviln mining and everything with me all this sultare that does that has alwaysseemed like. I feel like the switching and overhead coasts are way higher than the benefit is ofactually switching like what is the actual costs s associated withswitching, and does it still make that much sense to consistently be switchingbetween which cryptal crency you're minding yeah? No, that's a greatquestion and, and you absolutely right, there is a opportunity cost involved inswitching and that needs to be modeled into your crimars. It's it's actuallymore perplexing than fulks think it could be anywhere from how you actually switched the miningand if that incures Ti incurs downtime on the machines right, so you mightlose Hih rate for a couple of seconds that needs to be modeled in Um. Youalso need a model and h the ability to liquidate the orderson the exchange side. So I think your point is a hundred percent accurate.That is not done correctly. You might actually end up hurting yourself morethan helping yourself, but if you do it correct, you cansactor all those prameters in and still make a decision. That makes sense froma financial point of view. We've been running this Um. You know, since I'vebeen here and Um, you know: We've seen pretty substantial, lift we're onlyswitching between three coins right now so BCHBSB ANDBC, and we did explore PPCat one point, but it has so much hair around consensus and and gettingconfirmed blocks that we, we jetisoned mining with that. But those are thethree we switch between Um and, you know, truth be told a lot of theofporgunities between those really come in the the fact that Bitcoyn has obviously atwo week, difficulty varians and then the other coins have you know AlmosTransaction base, so that creates these gaps between Um, a slow difficulty,change on Bitcoyn and a very dynamic difficulty change between the other twocoins, and that creates these spreads that if you're smart, you can takeadvantage of that makes sense as to why you try and start with those 'cause. You have veryspecific, minding cardware that Um and then the underlying agrithms thattake advantage of that hardware are a pretty one to one mapping and they'renot allow of variants across the different different chains en you have. What so like you have. Youhave kind of this large variance cross. How difficult tiy change this processthat works and guess associated price fluctuation. So training on that alone makes sense,whereas trying to bring in other things like etherium GPUMAS Ciin across the board,which is a myriad of other coins. Rastically increases the amount ofthings you have to adjest and and make decisions on correy correct. You know,go ahead Ike! Is that something Yo lo looking into getting into, because Iknow you do more than Asic minding it at of course, scientific yeah. So mostof our fleet is a sick in shot to five six BAS Ger. We do have GP customersand have GPU Geare, but you you know you hit the nail on the head. You onceyou're Goig into GPU, and you start switching AUGORITM that you know youchange the power consumption of the gear and these quains have some of them has such low liquiditythat it's really hard to. You know you might make a a calculation that said coin is themost profitable and then, when you trying go and clear and order onexchange, you can get enough buyers, and so you know we're trying to stayaway from some of the fringe exchanges and we trying to focus on coins that webelieve have enough liquidity to be able to do the actual liquidation aswell as just switching Um, the you know under GPU side, there's a ton, moredifferent coins, Um and you K, but th I don't feel there's the same gravy TiSaround. I S A few coins on the GP side, that's worth exploring, but we havelooked at it oans your question. Nd He's, definitely looked at it and ouralgorithms can consume, and these signals Um. But, like I said, themajority of our fleet is sho to five.
Sixty the OSE CIMMA exquession is like Um Wat all like the types of things thatyou are ingesting as signals for these things to make decisions on. It seemslike, as this is mentioned earlier, the Biin Po gast episode, MHM all the waydown from computational power. Energy price market signals social data so on and soforth, heeause AE tremendous on. He things you can adjust. What are you?What are you seeing is like H, the most useful mhmadgestionindicators, FO figure, new things out yeah. I wish I Iwish. I could tell you some Sur, interesting story of you know it'stit correlated with you know the amount of Blah Blah Ba. You know it's realityis that we we do look at a lot of signals, butyou know the good old trade volume is still one of the strongest drivers M interms of a feature in predicting Christ. Um, the you know, we look at our firstparty signals. Obviously managing a fleet the size of what we do gives us aton of direct signals, as you mention for cost, on the Caut side with tower and and then we look at some third partysignals. Obviously, exchange prices, UM and difficulty block rewards. You knowthe usual ingredients. You would put in place to calculate croftability, but heans your question. You know, at least in our experience, one of the largestyou know to. If you di the principal component, andtathis is the biggestcontributor to our forecasting Israely trade volume. There are some macro trains like therainy season in southwest China. That does play a roll, but we we don't modelthat in our models we just kind of do that as an overarching adjustment. So I guess you mention predictions andlike a Lo, I mean it's quite popular, a popularopinion that that Crypto is voluntile and not very rational dos that so like judging by your models and yourpredictions, do you agree with that statement like?Can you actually predict things re reliably enough based on these inputs that you'retaking which, as you said, is like volume and stuff? Are these actuallygood indicators of how the price will be affected in the future? Like I say, O like like, what's thetime Rize in the tmost predictions, 'cause th the get work toworseis thatyou go like as you go further and further to the future correct. So ifyou guys, if your audience could see me, they'll see the grey hair from tryingto predict cryptoccurrencies. Yes, it is completely. You know it. We, you knowthe the predictions we do are short tem and that allows us to mostly try andforecast whether we believe or on an upward trend on a currency or adownward train than we use that to set the order floor prices. So we believethe bit coin is going up we're going to try and increase our H, clearing prices,onder exchange to just above ask, so we can kind of step up everythink therebest. Obviously we go slightly below Um, but that is but that's about a sevendayhour forecast two to seven nayouts. Once you try and go further, it's it'salmost in crazy town, it's th, there's! So many weird factors that drive theChrist- and you know you just can't get signals around some of these. Somegiant whale goes and sells a lot of bidcoin. You know the signal, you get astraid Vogume, but you don't get anything ahead of that. So Um. If youdo a short enough, rediction Um, I think you can still get utility out ofit. But you know: We've tried to run models out further into the future andit's pretty difficult to get a good, read pret, but that they said like how do you seethe marriage, ofmaybe, cryptol, currency trading or prognostication ofprice m evolving overtime? wher? Do you see it going? You see it going until Ikincorporating more and more assets. You see it intrying to narrow down different sources, bettermachine learning techniques, better computation for like Um trained modelsfaster like what do you see like t e, when you think about the advancement ofmachine learning at ai as it pertains to the work that you use it for? Butwhat's the next like? What are you excited about yeah, so so real, quickon the block chain side and then I'll...
...switch to just peer play, ison N, theblockchain side, I think Um, you know the one approach is to go after moresignals. But again, I think there's some events that impact the pricing that youwon't have signals to. Unless you somehow you were able to see into theminds of somebody that decides to drop a bunch of coin. On the marketplace Um,so you know, I think there it's really about the coins themstelves stabilizingand becoming Lek, less volatile, which they're going to have to do anyway. Ifthey want to, you know, be a quite UNQUITE FYSCAL currency um on the onthe per playiiside, I think Um, the sky is the limit. I mean we we'veobviously work with net uppen in video on Cov at nineteen research. Um, youknow AI, to go back to my earlier comment. Ai Is, is a suita his term.That means a lot, so you know you could you could consider ai to be anythingfrom a simple linear egression in aspreadsheet which a lot of companies are doing when they do sales forgam andthen g slightly more complicated and say: okay, deep learning and the twoprimary uscases there are computer vision and natural language processingUm. On the computer vision side, we seing a lot of companies, use that fromany anything in the lifescienceas. We see computational pathology, where usingmachine learning models, computer vision to diagonose Um we're seeingmanufacturing the effect detection. We seeing Los Lost Prevention Um. Youknow the most reason we working with thiscompany in Europe. Most recent USCAS, which I thought was pretty interesting,is and apparently it hasn't hit the? U S yet sodon want to give people ideas,but in Europe they're stealing these small atms that you find in like aseven eleven and so the way they do it. They draw little halls into these. Youknow self enclosed units and then they inject some sort of gas into it, andthen they ignide this gas and that's basically how you crack open these t,these atn boxes, and so with these guys this company's doing they are basicallydetecting these canisters, and so whenever they see somebody in the ATMcamera- and they see these cannerses come out- they you know, notify thepolice so Um. You know, there's very interesting use cases for catternrecognition in Computer Vision, Um and then obviously you have the slew ofother usecases anywhere from crowd control social distancing on thecomputer vision site, natural language processing is probablythe next biggest there's a lot of innovation happeningthere. It's much large and models being trained. So you think about birds, gpttoo Um, and that those usecases are from arobotic process, O automation,chatlots, so Wenave youinteract with a chatbots, that's natural languageprocessing. It uses a language Mogel to figure out how to speak back to you, UmThereis, also, obviously, the coal cinners we working with a customerthat is doing enormous amount of speech to text and what they're trying to dois when you call into a call center Um, they wantto to listen to the customerconversation and say: Oh, this is a conversation about X Y Z and then beable to do ai models to come up o recommendations for the for the callsinter person to say. Oh if the customer's talking about this devicethan these are some of the issues we've seen before right, so really H, helpingclosed at cause that knowledge, Gafti, Yo Wol on the coal center side, Um. Sothos are you know, kind of the three big blocks when you look at what peoplethink of as ai as the in a cyberdyng terminator completely autonomous entity. I thinkthose it's referred to as general artificial intelligence. I have notseen that for a while skind a while for that one, and if that's what you mean with aiadoption, then it's very low. If is the other two categories is what you meanwith Ai, then I think it's pretty ubiquitous um and um and it's you knowit's being woven into a lot of softar like if, if you look at your office assweets or spread sheeds, they all now have little widges that help you with.Oh, are you trying to do Xyo, Z or hey? We think you're trying to write aletter. Do you want to Blah Blah Blah? So that's all effectively a I ormachine learning.
You know something that I think a lotof people don't quite understand about that even further than the differentTuasion you just made is the fact that these models aretrained m and then compacted in away and then andthen they liv on the Clintside. So Tha t they're not like whul. You do Ese forsome of these things, like think about, like some of th, the text recommendation foryour chat or h how your seriy talks o you things like this. A lot of theTimes. The actual processing and recommendation is happening locally onyour client and not communicating with some central server. That's then doingit an Seatin it back to you and that's, I think, a big part of pushor innovation in recent machine learning, N Ai is is getting thesethings to be useful in such a compat size that they can be done on mopbaldevices and so on and so forth. Is that am I interpreting that correctly yeah,absolutely and and that's the other Pavit of of conoquat, a machine,leaning, there's the model training aspect, which is where you need the bigI n Thevice, says: G P, use lots of power, Luch of data, lunch of Crincrunching numbers and then they produce. You know, let's take an image nethundred and eighty gigabytesaimages and it maybe produces a model. That's youknow ten megabites Um and that, and that en megabized model thenneeds to be deployed on the infrencingside or the edgeside, whichis where most of their real world applications happen. So you're right.If, if it's a chat, Butt, the motor was trained in one facility on one set of Carnmary,but when you're actually chatting you you talking to the infrencing side ofit and and you're right, you run that on the edge it needs to be contacted quantities as they say when you reducethe precision of the weights of the model,to get it to fit on smaller gear, and you know you're looking at differentheart, where you're looking at potentially F PG A accelerators, ves,General Grippas gpus on the training side, but you're R right on that'sthat's another key th m. That AI is maturing is that it's moving awayfrom lar projects, which is mostly training, model and testing accuracy ofthe said models to actually product ion deployments. Where you actually putthese these models, ininto work and in those gas, it's all about entrancing,it's all about the edge. Is there something that Youare mostinterested in across, like Li, like you said like Ai, is a suitcase term, itshav een, incredibly broad spectrum of like very, very deepspecialities like Simiar like web development or data scientists, or youwhat have you? What other SUITGAS ERS TS associate the same type of stuff? Isthere something that you're specifically interested in MHM yeah?You know that I think there's kind of two schools. I thought the one is thatAI eventually replaces abot a lot of what we do, but I think a long timebefore that happens. It will enhance what we do, and this could be anything from. You know going back to MIT teminatorreferences and I'm not a xeminator fan boy, or maybe I aif you recall whenthey showed how the computer was looking at stuck how it had all thesehints show up. So imagine a world where you havewikopedia and your you know, wish beg and cold, and whenever you look atsomething against recognize and you can look at a car and it'll tell you theMormin, you can see all that so its this. This concept of augmenting what we do not replacingwhat we do or if a doctor is doing surgery, they have all these capy eyes,almost like a heads of display available to them or if you working in medicine Um our ability is humans to recognizepatterns is phenomenal, but the amount of data that's needed these days. Youknow AI is just great at recognizing patterns that and if you look at thework that googles doing with deep mind and other spaces where they can look atan Irish scan and diagnosed diseases, which you kN as phenomenal, that thesesignals exist. We just weren't able to recognize these patterns, so I think aiis going to help a lot um by making us as humans, better stronger, faster and then you know they will be. I think,quitewhat jobs that that AI will subsume as it gets,smarter and more and General Curpose, but Y. I don't know if that's in ourlifestymes or our kids lifestymes, but what I do know will be in ourlifestymes is ae that makes us better and makes us more efficient and helpsus understand. STHEF deeper, an and more thoughtful. I definitely see thatAndogreth it quite a bit.
U Kind of see here t that's. I wouldcall out of marriage of like if I think about the current expentialtechnologies that are in our hands. It's it's more of a marriage withmachine learning, a e with Arlik M that was augmen reality in firtualreaoowe a and so because of that you see those twothere's an obvious connection between those two technologies and how theykind of work well together. And then, when you start to bring that back into what we're trying to do in the blockchain space it it's it's kind of try. I guess preventing the distopiant Um manipulation of that world once itexists. So like say, if you partic imagine a world in which you have of good portion of how you view theworld and make decisions and interpret things is done through the assistanceof some type of pogmented reality that is taking advantage or taking advantage ofa lot of um machine learning, abrithms that are that a tere placed thanwhatever software are using right. How those things get created and howthey ingest data and then how they kind of speed it to you, Um ar all kind ofinterpoints into mandipulation ymy ship Erform, and so Ifeel as though like a potentialy usecase, for whatever we're trying to do in theblocktens bases, trying to give shrunger guarantees around that data,whether it be what you're using to train underlike model or s or Um. I guess the methodolgs May Qik upsoever R give out whatever whatever it's doine is that that of reasonable Apeik? Is that kindof what you'retalking about earlier? In terms of you need to have real strongguarantees about the data we're using to do these things 'cause. I H when I was socking on H, bicinpacasts.We ere discussing the importance of price data and the feeds that Y thatyou're ingesting into kind of what coins at what price and historicalthings overtime that inform you on what to then mind and make decisions on, and that's a very important thing thatcannot be vanipulated if you're going to make a good decision. atthat a like.So that's the kind of the underlying workal issue that kind o comes up, butif we just expand that across the board, um having very good data is a veryimportant thing and how that data gets Um, store, verified and then move tothe things that are actually going to use. It is incredibly important, Ithink, that's where bloc Cha maybe comes into play. Yeah! Absolutely!That's! That's! So, if you imagine going back to that augmented reality orUm, um t the the because l machine leaningcan learn catten so fast. You actually be rigt. I to have broad application ofthe of this augmented reality or th ability toargment. What we do, howwe understand things and all of a sudden you have a lot of people runningon models that were biased one way or the other. I mean it's, it's it'spretty dangerous in a way that you could have broad biases, bereplicating out by models that were training on bad data and Um. It will beso worten into what we do. It'll be hard to detect. Um, you know one of myfavorite books to plug is t thinking fast and slow Um and it talks about human mind's ability to almost not be able to distinguish truthas long as it's ubiquitous so basic. If doesn't matter how crazy an idea is, ifyou have enough social afformation of that concept, you would experience itas full on truth and so go take that with biasd models, and you really cratethis short. So good were Um. You know you could have really things go reallyhay wire, and so because of that, I think models are going to become almostlike Crypto, keys or stuff that get managedand trained by groups that have made sure that th, the modols th the classesare balanced within the models and if you do have models that are deinginfrencsaying on onlive data, there's a lot of technologies that look at calleddata drift. Were you just making sure that your incoming datasats fin varyingso much from thi stuff that was trained on that you're going to start making y?U Know Pretty much forhendom random prediction, so yeah, it's it's garbageand garbage out on steroids with with with machine learning, so you has to bevery careful hm that'sinteresting MHM, so you mentioned the sharing of modelsand models would be managed by a group of people. Is there already examples of thishappening like open sourcing, AI models,...
Yeh deanis already yeah? Absolutely so?U Most of the big models are either trained by academia or some of thelarge companies. So if you look at you know the first, you know majornatural language processing model Alma that was trained by the islandinstitute. Birds was trained by Google GPT UM andMicrosoft did a new one e transformer model. That's the largest model so andthese models are shared and then whath most practitioners do as they Um. They then go ahead and M Chun these models. Very few companiesactually take or image net, which is a famous computervision model. Very fewcompanies start from scratch. They usually take one of these models and,what's called Tunit, find tunit where you saying okay, this model knows howto detect visual objects, but maybe it doesn't know how to DETEC. You know aboat, for example, just making this up, and so you can then tweak that modelslight you to be able to Lernto TETEC parts, and so yes, tens. Your questiona lot of the bigger models, for you know, power and consumption and costreasons are done by a few companies or academia, but then they are used by alot more. You Know General Purpose, folks out in the field and so in a way,that's already happening Um, but you can imagine that that's justgoing to get more and more and more udiquitous as new types of models. NewPatterns need to be recognized, comes alone, itsoff. The Wall, like five years ago, I'd publish thepaper, an quonum computer, about Naqua Computin,but quatumdynamic calculations. Working with some, like modeling quatomicatics,on classical computers D, I had adapted Um neral networks to be used as a computationally, efficientinterpolation system so, like I guess, Xist to give quick prefix. There's these incrediblylarge complex, multivariate services that are required for these types ofcalculations on super computers and every time you evaluate one of thesefuctions that has a lot of variables in it. It takes sycnifical amount of timein order to do the calculations, these things seem to be Um. thise functionsneed to be evaluated millions of times right, O right and they become somewhatof a limiting factor and how long it takes you to get the job complete on onthes super computers, which is incredibly expensive, and the gold ofthe paper was to use mmachine learning, specifically nerl networks to create a um O'm, looking for basicallycomputational officient function that gets evaluated very very quickly,based on the large. The large surfaces is that something you see in the ecosystem in in other ways likeusing neral networks as interproelation engines, as opposed totrying to prognosticate based on some Corpus of data yeah I mean, I, I think it's tractable, that we getmore adoption. I think if you look at at the state of some of the modelarchitectures I mean we're still seeing new model architectures come out, andso I think wee far from having dialed in exactly what the the best activation function is o, whatthe best architecture is and should you do recurrent or convolutionoroccombination or ou K o. So I, I think, there's a lot as we can still learn andlot of new applications. foriearold networks Um the I think, as we as we get more the real world applications. Ithink it's going to require different ways of thinking of models. You knowthe one thing that was quite interesting. Speaking of and out of the box, usecasewas Um. The Rice University did a Hash based supplementation of deep narrow,Nas called slide slide Um and they basically deviatit from Geoffreyhinton's back propagation approached to figure out the weights OFAN our on that, and so that was a completely differentapproach. U It doesn't require gpus s, it doesn't do a lot of Matrixmultipication. Instead of that's a lot of Hash, stable lookups and you I can imagine that with theinnovation in hardware and in you network, architectures that you knowthings are going to get, you know we'll...
...be in five years time, we'll be lookingback at some of these models and go. How remember when we used to useIMENTIONAD or continents Ow, you know how primitive those were. This is whatwere doing today so yeah, I think, there's definitely going to be newUSECAS. Is that that come out that's about. I could probably talk about this for awhile, but h without divingo going one too many tangents is there? Are thereany questions that you wished? I would have asked that I didn't N. I think you guys th great, I mean Um, I'm really excited with with the AIspace. I think it is a transformational technology. It's anraing said it's like electricity and Ibelieve that it it's going to be transformatial. It's going to be ineverything we do in one shapewill forum. It's going to be very powerful, whichmeans Yiu could be useful, good and bad, so you're Goingno have to keep tabs onit. Just like you do with any technology Um, and I think it's gonna.It's going to help us some solve some challenging problems that you know. Ifyou, if you look at the pace at which ai canlearn. This is the pace at which we've learned. I actually wrote an Arti ablockbost on this. But U know if you look at how evolution allows you to learn and how you couldonly learn through linage and now w. Then we started writing, and so youcould share stuff between folks and other people and learn and havememories. F. Oh, you know, don't stick your hand, an anlionce mouth, 'cause,it'll, bide it okay. That becomes something that you've learned fromFrolan Ara, but with with deep landing, we can continuously learn, and so youknow that that in n of itself is amazing of what we'll be able to do by having just continuously learninglearning learning observing recognizing new patterns, so I'm really excitedabout Ai, which is why I work in this field. I'm excited, but what Rol Corpplays and and empowering Deata sciencists to solve these problems, and so yeah I mean, I think you guysdid a good job is tsay. Anything else. Youwould want to ask o want me to digand more Um for your audience te M. I mean I there're so many stupidquestions. I could ask about Ai 'cause, I'm genuinely interested in the field,but Um don't know much about it. A D, Witaevi,stupid audience member and ask those questions. Okay, the les are with the the one which islike recent GPT, three right: what's the big meal, so wh n, when you get to these languagemodels, um it's you know, really t it's calledembedding word: ambettings Um! It's trying to learn the relationshipbetween things in a almost milked, dimensional graph and so typically W,whether it Pur gpt two GPT, three elmo the issues there is just on how manyparameters you've trained. If you look at the brain, I think it'sa hundred billion synopsis to think of those as parameters and so gpt three is just a very, very large model that was trained, and so theconjecture is the more parameters. The more complexion model is the more umadvance it could be in detecting and disguse language models, Um yeah, sothat that's basically the the thesis around around these m natural languageprocessing models, Soe Iger, is better, is gpt three, the most advanced oneright now. I think GE B, I I'm not actually I'mnot sure whether there's one that microsove racenly, that t something Icant remember the acronym, but they did a transformer BAE Model and I'm notsure if gbt three is actually larger than that. Okay Hat, you could justlook up the number of parameters like I said the reason this is limited tothese big companies, its really who has access to that much computational power to be able to train these models um butyeah. It's it's a toss up between Gbd, three and TPA or SOM, can't rememberwhat Microsos ones called then I'm I'm kindo surprised at likethe model. GPT three came out of Opena eye right,which is like it's the en must company. I think right,Ereyes I'. I'm kind of surprised that stuff like that, like is Google, doingopen source stuff like this 'cause. You think they have to vote thecomputational power and the datasets required to get something equally, ifnot better, off the ground yeah. No,...
...they they did M. I think gpt too wasdone by Google, I'm not mistaken. Okay, yeah theyso! You know the like thelanguage models. They all have funny names, Almoburd, GPT, two gbd three andthen micro, Microsoft, one we show the life of me. I can't remember Um so yeah they they do come out of eitheracademia or or o non profits. Open AI believe train their model and Microsotas your infrastructure. 'cause microsoven base id a good portion of of infrastructure into open ai, butyeah these. These are great grat for natural language processing and it'sit's funny. We've kind of Stope innovating much on computervision. I think people feel that that's at a reasonable place, and noweverybody is innevating. Onr natural language processing models, Um, and youknow I think what will happen is we will learn about new approaches that wemight circle back and go apply on computer vision, again, Um and so yeah.I think this is going to continue to we're going to continue with newarchitectures and Um and finding ways to shring them down to run an GEdevices. To that end, it seems really important to have people like you, ainfrastructure companies trying to figure out like what power computationdevices architectures to be have available to trade, these variousthings and weare they most efficient Lik. I, like you, said, like peoplecan't do these things unless they have access to a specific amount ofresources to do it and or or if those resources are even availablebroadcasting the people who would like to do them and without bring somee likeyou in your position, it's really hard to make to bridge that gap. Are youseeing Um your competitors, other people whoprovide these resources higher people who are interested in machinelearning research? Or is it just part of the deal like you? Don't run acompany like that UN Mest? You have someony like you involved Wel to the first part of your statement,absolutely tre. You know the the example that comes to mind as we did initiative with MIT in one of theresearchers crain. What's called the Big Gan, the generative adversarialnetwork on our infrastractor, and that was the first time somebody trained atnetwork outside of Google. So absolutely our goal is to help bringthis capability to to all data, sciencest, Um and M and yeah. We want it to bedemocratized to make sure that everybody can can have access to thisinfrastructore in one shape of form. It is actually much harder to manage thanyou then traditional. You know, dal severs example, and so there's a expertise aroundmanaging that around sharing them making use of them. The the worstCardinal Sen you could do is is by these expensive infrastructure machines,e servers and then not be able to utilize them and have you know idle GPSor fifteen percent utalization, especially since you pay such a hiftyprice for them. So we make sure that peopl can utilize their infrastructurethe best possibleay. It makes no sense that Tha it remind that Sind me back in almostptsd of handling H, job qes on Supat or sciencificapplications. T it's like. There's no much congun power not being used rightnow. What's going on here right, exactly it's brutal! You know the this.This researcher made a comment. The the challenge, of course, is unless youhave fully accelerated fabric right everything from storage, a not workingto the th. You know where they use NV, linkendreswitch opce expred that wholepipeline of getting data onto that silicon, onthe GP, has to beaccelerated or you're going to end up with under utilized infrastructure. Andyou know you, you have ten percent under utilized CHEP Yeu. That adds up alot and so with our with our softre caled plexes. When you run a workload, it will make a recommendation to you tomake sure you're not going to run on GPS that are going to be half utilizedand you're going to pay an arm in the leg for ONA public crovider, forexample, it will. It will riht Sizei infrastructure for you to make surethat you're, not overpaying and or if you runt a job it'll tell you listen.You have ample C bu cycles to spare in Ow, maybe consider moving some of thecompute onto the CPUSO. Just you know to the midapoint of how do you Mamaximize your investments and make sure...
...you don't have idle cycles, becausethese these you know as as amazing as they are in video releases in EU GParchitecture every two years? And so you need to make sure you get the mostout of that and that's you know, that's something that th the mining folks now,Lo Ogs, you know when I started with Cor, snines wet alltheir rage and Um. Now you know as nines are struggling, and so you haveto maximize the value you get out of this infrastructure in the two yearwindow of which it's Ou know considered steade of the art and that's anotherthing that we see an overlap between blacchin and AI HM. That's really interesting! I co.That's, probably all what the episode in itself is trying to discuss the thevinkage between Hardwar architectures in the Software Tho running them andthen how you figure out Rahow to maxilize that yeah 'cause it I it doesit's not like yo KC. I mean I think my pieces. Well, I just got a new one, butthe one I had before that was you know eight years old eight year old GPU is Antique Andof. Your old mining is thxest right. So it's a whole. It's you knowit's make the most. While you have that kind of game. Aright Didn, you geve anything elsehere, no riht. Ah, I really POCIAT Youe Cominkind of chawith. U About this stuff! It's it's quite fastane te dig on onthis side of the conversation which o don't hear much of so nextr ex comingon. Maybe we can hav Yo back to drive into Porstelf sans great thanks.Everybody.
In-Stream Audio Search
NEWSearch across all episodes within this podcast