Re: [go-nuts] force garbage collect before crash

[Robert Engels]

Makes sense. Thanks. 

> On Oct 18, 2020, at 9:13 PM, Ian Lance Taylor  wrote:
> 
> On Sun, Oct 18, 2020 at 1:24 PM robert engels  wrote:
>> 
>> If I set a low process memory limit, and then the OS refuses to grant more 
>> memory to a malloc request,. will the Go runtime “stop the world” and 
>> attempt a full garbage collection and try to get the memory from the heap 
>> and then only crashing the process if it can’t?
>> 
>> I understand that the “pacing” should limit (eliminate?) this situation, but 
>> I am thinking in cases when the heuristics don’t match the immediate 
>> allocations it would be beneficial.
> 
> It does not work that way today, no.
> 
> The current Go memory allocator allocates address space and then
> expects that address space to be available.  Typical GNU/Linux systems
> use overcommit, so there is often no system call reporting that memory
> could not be obtained; instead, there is a SIGBUS signal when the
> program first writes to a page that the kernel can no longer supply.
> It might be tricky to safely run a garbage collection and then retry
> the write.
> 
> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1DA916A6-B365-4C27-9CC8-8C6E155B0CD0%40ix.netcom.com.

Re: [go-nuts] force garbage collect before crash

[Ian Lance Taylor]

On Sun, Oct 18, 2020 at 1:24 PM robert engels  wrote:
>
> If I set a low process memory limit, and then the OS refuses to grant more 
> memory to a malloc request,. will the Go runtime “stop the world” and attempt 
> a full garbage collection and try to get the memory from the heap and then 
> only crashing the process if it can’t?
>
> I understand that the “pacing” should limit (eliminate?) this situation, but 
> I am thinking in cases when the heuristics don’t match the immediate 
> allocations it would be beneficial.

It does not work that way today, no.

The current Go memory allocator allocates address space and then
expects that address space to be available.  Typical GNU/Linux systems
use overcommit, so there is often no system call reporting that memory
could not be obtained; instead, there is a SIGBUS signal when the
program first writes to a page that the kernel can no longer supply.
It might be tricky to safely run a garbage collection and then retry
the write.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVLtwegxAONSqfNreBhB7tz9nOo1PfALN2OsXaZOdEoHA%40mail.gmail.com.

[go-nuts] Re: I need help writing an article about the Go implementation of hash tables

[peterGo]

GopherCon 2016: Keith Randall - Inside the Map Implementation
https://www.youtube.com/watch?v=Tl7mi9QmLns

Peter


On Sunday, October 18, 2020 at 9:10:45 PM UTC-4, Marwan abdel moneim wrote:
>
> Hi,
>
> I have some free time, so I thought about writing an article about maps in 
> Go, and how it is implemented. It would be a good dive for me and for other 
> people.
>
> I came across this file 
>  in the 
> source code. But while I still remember what I studied about hash tables, I 
> find it very complex.
>
> So how should I approach this task? Should I read something before reading 
> the code?
> How does a new developer on the Go team get to understand a part like 
> that? 
>
> Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/553824de-4689-43f7-a1d5-0494a43cb135o%40googlegroups.com.

Re: [go-nuts] I need help writing an article about the Go implementation of hash tables

[Ian Lance Taylor]

On Sun, Oct 18, 2020 at 6:10 PM Marwan abdel moneim
 wrote:
>
> I have some free time, so I thought about writing an article about maps in 
> Go, and how it is implemented. It would be a good dive for me and for other 
> people.
>
> I came across this file in the source code. But while I still remember what I 
> studied about hash tables, I find it very complex.
>
> So how should I approach this task? Should I read something before reading 
> the code?
> How does a new developer on the Go team get to understand a part like that?

We can answer specific questions, but it's hard to cover a general
topic like that.

That said, I suggest you watch this talk by the author of the current
map implementation: https://www.youtube.com/watch?v=Tl7mi9QmLns .
It's a little bit out of date, but not much, and should serve as a
good overview.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVqnUYsYiJrLy9KGXXGG0cMs1ZY8yHLu_8a1Wf_d0_Vaw%40mail.gmail.com.

[go-nuts] I need help writing an article about the Go implementation of hash tables

[Marwan abdel moneim]

Hi,

I have some free time, so I thought about writing an article about maps in 
Go, and how it is implemented. It would be a good dive for me and for other 
people.

I came across this file 
 in the source 
code. But while I still remember what I studied about hash tables, I find 
it very complex.

So how should I approach this task? Should I read something before reading 
the code?
How does a new developer on the Go team get to understand a part like that? 

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0378e35e-2ce7-4a29-b946-c9a2e5c1e372n%40googlegroups.com.

[go-nuts] Free Go Getting Started Course

[Michael Levan]

Hi All,

I recently completed a free Go mini-course on YouTube. It's nothing too 
long, just 7 videos. However, it'll help you get started with Go. If you 
know someone who may need it, feel free to pass the playlist along :). 
Cheers!

https://www.youtube.com/watch?v=P8Jx3f01igA=PL8iDDHqmj1oUHd2g0iYWHmNLSZPVdigC_

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d178bad4-4919-4e5f-a91b-5d9fdc217e34n%40googlegroups.com.

Re: [go-nuts] net/http TLS issue

[pat2...@gmail.com]

On Sunday, October 18, 2020 at 4:39:46 PM UTC-4 Rich wrote:

> I have no problem writing this myself and I already have ideas on what I 
> am going to do, but before I do that I thought I'd ask and see if anyone 
> knew of a package, or some easier way that did this already. I've spent 
> hours in the past writing code only to find later someone has written a 
> great package that does what I need and more. Never hurts to ask right ;
>

The great packages have been written. But the real solution is to use a 
service either in software like AWS Secrets Manager or in a hardware box 
like the banking networks like.  If your company has had this policy a long 
time, use whatever other systems have used. 

Writing crypto software that seems to work is not hard. For this case, have 
the operator enter a passphrase, run it thru a HMAC, and then use AES 256 
to encipher/decipher the CERT. Store the binary in some standard format, 
try to avoid ANS.1 if you can possibly do it.

Writing crypto software that really works, is secure, etc. is very hard. No 
offense, but you are unlikely to do it.

I don't understand how this is supposed to work. Do you ask the system's 
operator to type in the passphrase every time the service is restarted? 
Most modern systems don't have a human operator to do such typing. And how 
do you ensure that there is no keyboard logger on the operator's terminal? 
Or are you keeping the passphrase in someplace like an environment variable 
or a secret file?

The software and the crypto are only a tiny part of the issue. Either do 
whatever made them happy in the past, or find out what is really driving 
this "requirement".

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0030732d-3fd2-487a-99dd-09395ee9f894n%40googlegroups.com.

Re: [go-nuts] net/http TLS issue

[Rich]

Thanks for your reply, however as I stated it's company policy to have a 
passphrase, even our java. keystores have a passcode. Using other services 
or methods, or just not having a passcode -- Just not the way they want it 
done. I am not a part of the security team that makes this stuff up and you 
might as well try to get water to flow up hill to change policies. It is 
what it is, I gotta deal with it. 

I have no problem writing this myself and I already have ideas on what I am 
going to do, but before I do that I thought I'd ask and see if anyone knew 
of a package, or some easier way that did this already. I've spent hours in 
the past writing code only to find later someone has written a great 
package that does what I need and more. Never hurts to ask right ;)


On Friday, October 16, 2020 at 6:33:14 PM UTC-4 mar...@gmail.com wrote:

> Having a passcode to protect a key file for a production service is 
> pointless, because you move the problem of storing the certificate securely 
> to the problem of storing the passcode securely, so might as well skip the 
> passcode and store the cert securely.
>
> Your certificate is probably encoded as a PEM cert, so you'd have to 
> manually call https://golang.org/pkg/crypto/x509/#DecryptPEMBlock and 
> provide a password, then construct your listener yourself using the 
> unmarshaled certificate. So, how are you going to protect this passcode? Is 
> someone going to have to provide it every time you start?
>
> Generally, in production systems, we use some kind of secret manager to 
> store that certificate, such as AWS Secrets Manager or encrypt it with KMS, 
> store it in Vault, etc. Ideally, you actually make a subordinate cert for 
> that service and rotate it at a reasonable interval.
>
> On Fri, Oct 16, 2020 at 2:06 PM Rich  wrote:
>
>> I don't know if we're the only company on the planet that demands the 
>> https keys have a passcode to them but I've been having a heck of a time 
>> trying to find a good way to serve https using a key with a passphrase.  
>>
>> err := ListenAndServeTLS(addr, certFile, keyFile string 
>> , handler Handler 
>> )  
>>
>> If they keyFile has a passcode this doesn't work and the examples I've 
>> seen take this one line and turn it into a much longer chunk of code.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/c75fc8f6-abe4-4614-8281-cef4cb315ac3n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/732f64f7-d4ae-49f3-be69-58a816005097n%40googlegroups.com.

[go-nuts] force garbage collect before crash

[robert engels]

Hi all,

If I set a low process memory limit, and then the OS refuses to grant more 
memory to a malloc request,. will the Go runtime “stop the world” and attempt a 
full garbage collection and try to get the memory from the heap and then only 
crashing the process if it can’t?

I understand that the “pacing” should limit (eliminate?) this situation, but I 
am thinking in cases when the heuristics don’t match the immediate allocations 
it would be beneficial. 

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6918A9B9-F98C-4B74-9B66-2212F531FCC7%40ix.netcom.com.